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National Assessment of Educational Progress (NAEP) and the trustworthiness of 
the scores that the states received during the 1998 reading assessment. It 
addresses whether Vice President Al Gore’s "release" of the results violated 
the apolitical nature of the NAEP, and whether or not including students with 
disabilities affected the results of the assessment. The document presents 
the opening statements of Subcommittee chairman Pete Hoekstra, and an opening 
statement of ranking Subcommittee member Tim Roemer. It then presents the 
statements hearing transcripts of Pascal D. Forgione, Jr., Commissioner of 
Educational Statistics, and Mark D. Musick, Chairman of the National 
Assessment Governing Board. Appendixes contain the opening statement of Pete 
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REVIEW AND OVERSIGHT OF THE 1998 READING 



RESULTS OF THE NATIONAL ASSESSMENT OF 
EDUCATIONAL PROGRESS (NAEP) - 
THE NATION'S REPORT CARD 



Thursday, May 27, 1999 

•House.of Representatives 
Subcommittee on'Oversight and Investigations 
Committee on Education and the Workforce 



The Subcommittee met pursuant to call at 9:30 a.m. in Room 2175 Rayburn 
HouseOffice Building, Hon. Pete Hoekstra, Chairman of the Subcommittee, presiding. 

Present: Representatives Hoekstra, Schaffer, Tancredo, Fletcher, Castle, 
Goodling,-Roemer, Scott, Rind and Ford. 

Staff Present: Robert Borden, Professional Staff Member; Becky Campoverde, 
.Communications Director; Victor Klatt, Education Policy Coordinator; Patrick Lyden, 
Legislative Assistant; Michael Reynard, Media Assistant; Deborah Samantar, Office 
Manager; Rich Strombres, Professional Staff Member; Kent Talbert, Professional Staff 
Member; Christie Wolfe, Professional Staff Member; Cheryl Johnson, Minority 
Legislative Associate; Gail Weiss, Minority Staff Director; Mark Zuckerman, Minority 
General Counsel; Alex Nock, Minority Legislative Associate; Cedric Hendricks, 
Minority Deputy Counsel; and Roxana Folescu,Minority Staff Assistant, Education. 
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Chairman Hoekstra. The Subcommittee will come to order. 



Pursuant to normal Subcommittee operations, opening statements will be limited 
to the Chairman and the Ranking Member. All other statements will be submitted for the 
record without objection. 



We are doing pretty well. The last Hearing started early. This one started about 
three minutes late. So, on average, that is about two in a row. Let me give you my 
opening statement. 



OPENING STATEMENT OF CHAIRMAN PETE HOEKSTRA, SUBCOMMITTEE 
ON OVERSIGHT AND INVESTIGATIONS, COMMITTEE ON EDUCATION AND 
THE WORKFORCE 



We are here today to address the potential politicization of the National 
Assessment of Educational Progress (NAEP) and the trustworthiness of the scores that 
the states received during the last reading assessment. 



As the nation's report card, the National Assessment of Educational Progress, or 
NAEP, is an ongoing, congressionally authorized program that collects data through 
surveys of the educational performance of students in America's elementary and 
secondary schools. The national assessment is carried out by the Commissioner of 
Education Statistics under the policy guidance of the National Assessment Governing 
Board, orNAGB. The Board was created in 1988 upon the recommendation of the 
Alexander James Commission, which urged that policy making for the national 
assessment be buffered from manipulation by any individual, level of government or 
special interest within the field of Education. 



In 1990 NAGB adopted its policy statement reporting and disseminating national 
assessment results. This policy statement was and in my opinion still is, straightforward 
and unambiguous. Specifically, it states that "Reporting and dissemination of national 
assessment results shall be insulated from partisan political considerations and 
processes." In addition, "The public reporting and release procedures for NAEP shall be 
apolitical, similar to those of other congressionally mandated data gathering programs. 
This policy is necessary to ensure that the results and indeed the entire testing process is 
based on independent judgments that are free from inappropriate influences and special 
interests. 



On November 6, 1998 the National Center for Education Statistics submitted its 
release plan for the 1998 NAEP reading test. The release plan stated that the press 
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conference would take place in early February, 1999 and would begin with introductions 
and data presented by Commissioner Forgione, followed by comments from Secretary 
Riley and a member of NAGB. This release plan was consistent with board policy that 
stipulates that "Reports of NAEP results shall be issued by the Commissioner of 
Education Statistics." It was approved unanimously by NAGB on November 21, 1998. 



On February 10, 1999 Vice President Gore was the featured speaker for the 
release of the 1998 NAEP reading results, the first time any official higher than the 
Secretary of Education had ever participated in a NAEP release. The Vice President 
commented on several administration initiatives during the release including increased 
funding for Even Start and Head Start, connecting every classroom to the Internet, funds 
for schools construction bonds, 1 00,000 new teachers, and even AmeriCorps. 



The point is not whether I am for or against the programs the Vice President was 
talking about. It is whether or not his speech was "insulated from partisan political 
considerations," as the NAGB policy stipulates it must be. In addition, it seems as if the 
Vice President was not telling the whole story about the test scores. He noted that, "for 
the very first time, reading scores have improved for each of the three grades measured 
by the National Assessment of Educational Progress, and this is great progress, and we 
are proud to report it." What the Vice President did not mention was that the fourth grade 
achievement level showed no statistically significant changes. Fourth grade scale scores 
were constant, and analysis from 1992 to 1998:in the 12th grade shows no net gain over 
the period. 



During the press conference, the Vice President made quotes about a down 
payment on hiring 100,000 new teachers to reduce class size. He noted the 
administration had worked very hard to help states and communities marshal the efforts 
of thousands of young people through the AmeriCorp program. He even encouraged 
Congress to enact the President’s agenda to improve education. 



The Vice President's participation was not an apolitical release of educational 
data, but rather it seems it was an orchestrated media event to promote a political agenda. 
E-mails from the Office of the Vice President described the event as "a VP education 
event" that was "good for the VP, for the Department and for education." 



The Secretary's office made suggestions about the program's dimension stating, 
for example “it is a reading day, why not push for reading dollars?” with regard to the 
Reading.Excellence Act. According to Mr. Musick, a witness before the Subcommittee 
today, "The format, tone and substance of that event was not consistent with the principle 
of an independent, nonpartisan release of national assessment data.” 



Indeed, Education Week reported last March that Vice President Gore jeopardized 
the integrity of the test when he announced the 1 998 reading results to a campaign-style 
rally. The Vice President's appearance as the featured speaker before the Commissioner 
differed considerably from the NCES release plan that was adopted by NAGB. 

3 



o 

ERIC 



8 



4 



Chairman Goodling expressed his disappointment because of the politicization of 
the announcement in a letter sent to Secretary Riley on March 17, 1999. In his response, 
the Secretary claimed that Chairman Goodling’s "concerns about the release of this data 
are un wan anted." 



With all due respect to the Secretary, I must disagree with that assessment. I 
share Chairman Goodling’s concerns and felt that the events surrounding the release 
warranted further review by the Subcommittee on Oversight and Investigations. The 
Committee has received materials from NAGB and NCES that reveal a blatant disregard 
for the policy statement on reporting and dissemination of national assessment results. 
According to information submitted to the Committee, Commissioner Forgione 
participated in a conference call on February 8 with representatives from the Office of the 
Vice President, the Office of the Secretary of Education and the Department’s Office of 
Public Affairs. It is our understanding that at that meeting Commissioner Forgione 
supported an agenda in which he would open the press conference and then release the 
results at 10 a.m., followed by the Chairman of NAGB. Finally, the Vice President, the 
Secretary of Education and a teacher would be given an opportunity to comment on the 
results. 



Subsequent to that meeting on February 8 and before the public release of the 
results on February 10, the Department informed NCES that the agenda had been 
changed. There was to be a shift in the order of the presenters and a delay until 1 1 a.m., 
at which time the Secretary and Vice President would open the press conference. At a 
minimum, the approved NCES release plan was disregarded. Subsequent to the press 
conference on February 10, NAGB had adopted the new language for their policy on the 
dissemination of results in an effort to clarify the necessity for the credible release of 
national assessment data. 



As I mentioned before, I believe that the original policy adopted by NAGB in 
1990 was explicit and unequivocal. When it comes to making important decisions about 
the NAEP test itself, NAGB should be completely isolated from politics. When it comes 
to measuring the progress of our nation’s children, we need to ensure that scores are as 
accurate as possible and we must not let politics influence our ability to measure student 
achievement. 



In addition to questions about the release of the scores, I have questions about 
how well the new NAEP inclusion policies are working. Why did the exclusion of 
students with disabilities and limited English proficiency increase since the 1994 test? 
Was NAGB acting independently and in the interests of getting valid test data when it 
decided to adopt a new inclusion policy for the 1998 reading test? Was NAGB pressured 
into making decisions that have brought us to where we are today, questioning the way in 
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which the results were released and the possibility that the scores themselves might have 
been clouded by politics? 

These are questions that must be answered to understand what we need to do to 
protect the NAEP tests from political manipulation. Specifically, we should review the 
process in which the Vice President became the featured speaker, discuss whether or not 
the Vice President's comments were thorough and consistent with board policy, and 
revisit the issue of whether or not it is possible for NCES and NAGB to remain 
independent as currently configured. 



I would like to thank our witnesses for appearing before the Subcommittee this 
morning. 



OPENING STATEMENT OF CHAIRMAN PETE HOEKSTRA, SUBCOMMITTEE 
ON OVERSIGHT AND INVESTIGATIONS, COMMITTEE ON EDUCATION AND 
THE WORKFORCE - SEE APPENDIX A 



Chairman Hoekstra. With that, I would yield to Mr. Roemer for any statement he may 
have. 



Mr. Roemer. Thank you, Mr. Chairman. I ask unanimous consent to revise and extend. 



Chairman Hoekstra. Without objection, so ordered. 



OPENING STATEMENT OF RANKING MEMBER TIM ROEMER , 
SUBCOMMITTEE ON OVERSIGHT AND INVESTIGATIONS , COMMITTEE ON 
EDUCATION AND THE WORKFORCE 



Mr. Roemer. Thank you, Mr. Chairman. 



Mr. Chairman, I couldn't agree more with you from what you have said at one 
point in your statement that politics should not influence this process. We should not 
have politics influence the process, whether it be a President or a Vice President. We 
want to know how our kids are doing on these scores and why they are not doing better. 
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I am a little disappointed, quite frankly, that today's focus of the Hearing is on 
how this thing was released rather than what we can do to help our children in this 
country do better only their scores. That is the purpose of this Committee, that is the 
purpose of the education oversight, and I think that is what the American people want us 
to do in this Committee. 



I was talking to a constituent last night who is a farmer, and her farm_her family’s 
farm is going out of business after 100 years in the hog farming business. She said, you 
know, you guys got a lot on your plate today. You guys in Congress have a war going 
on, you have got depressionary prices for our farmers, and you have got school shootings. 
You have got a lot of important things to deal with. 

I think we should be dealing with school safety. I think we should be dealing 
with how to improve in revolutionary and bold ways how our schools do better to help 
our children, and not politicize this process. 

I think that if we are not careful, GOP is going to stand for Gore: Obsessive 
Paranoia if we are not careful here. 



Now, let's get to the facts on all this. The Vice President may have made a social 
faux pas. Maybe he should have let Mr. Forgione speak first. But this information was 
released an hour before on the web. This information was out there for the general 
public. This is important information for the country to understand. 



And I am told from my staff, who does great research and always hits the mark, 
that back in June of 1992, there was a letter from the Commissioner, I guess, Mr. Musick, 
your predecessor, Mr. Boyd, who wrote the following, "in accord with that policy," he is 
referencing something that happened, "the Board approved a plan for the release of the 
1990 NAEP reading report at a press conference in Washington. 1 was surprised and 
disturbed therefore to learn that on the day before the scheduled press conference of the 
main results in the report were released prematurely by President Bush." By President 
Bush. 



Now, we can have a Hearing on how President Bush released these prematurely 
and speak about that. Quite frankly, I am not interested in that. 1 am not interested in 
President Bush talking about these NAEP scores at a private fund-raiser or school the day 
before. 



I think with Vice President Gore being part of the conference, maybe he should 
have said, Mr. Forgione, you go first. According to the statute, technically, you should 
be the first one to talk, and then I want to talk about these scores as well. Maybe that is 
the social faux pas. 
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But I don't think our next step in this Committee should be to have Tipper Gore 
come up here and start asking Tipper Gore, you know, do you raise your hand before you 
talk at a PTA meeting? 

I mean, let’s not be obsessed with politics here. Let’s get to what this Committee 
is supposed to do. How do we look at and assess these scores and their impact on our 
children and our teachers and our schools and how do we boldly improve our schools for 
all of our children in this country? That is, I think, what we should be concentrating on. 

Now, we can talk, I guess, all morning about what President Bush did or what 
technically Vice President Gore did. I am not interested too much in that. I am interested 
in, was this report accurate in what it assessed with some of the physically challenged and 
disabled kids? Was it accurate with respect to 14 States or two States reporting on those 
physically challenged kids? 

But I also want to talk, Mr. Forgione and Mr. Musick, about how do we get to 
some test scores in the next 10 years that we can look back on and say, our Nation’s 
children are doing better every single year. Our schools are getting better because of 
things we did in 1999 in education. That is what I am interested in. Not politicizing this 
process/not going back in history and talking about President Bush or Vice President 
Gore and how they may have made a social faux pas in releasing this data. 

So, I would hope that we would get to what you and I, Mr. Chairman, have 
worked on in bipartisan ways, in cooperative ways, in genuine ways, on a Hearing on 
school safety, on some Hearings on bold and creative reforms in public education, some 
Hearings on what Paducah and Pearl and Jonesboro and Springfield have done to make 
their schools safer in light of the tragedies they have gone through. Let’s get open to 
some of these very important programs for our Nation’s children and for our Nation’s 
schools. 



I yield back the balance of my time. 



Chairman Hoekstra. I thank the gentleman for his statements. I think you are right on. 
I am disappointed you are not all that interested in the integrity of the process. We have 
an agency out there that has rules and regulations, and they really need to be independent 
so that they can drive the debate. They provide us with the statistics so we can have the 
debate rather than becoming perceived as a mythical organization where we are arguing 
about the results which happens when we might cease that kind of activity. That is not 
what we want to have happen. So I am very interested in maintaining the integrity and 
the support that these organizations have achieved and that these individuals strive for. I 
think if we lose that, a lot of the other things that we have just doesn’t make much of a 
difference and we can't have a good debate any more. 
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Let's get on to the panel. Mr. Castle, thank you for joining us this morning. I 
understand you would like to introduce Dr. Forgione. 



Mr. Castle. Thank you, Mr. Chairman. I share the thoughts of both you and the 
Ranking Member in many ways. The process should be above any question whatsoever. 
If you are going to test, and believe me as somebody who was a Governor if your state is 
going to be compared, you want to make sure the test is fair and not politicized. Of 
course, we are addressing all the other issues as well. 



I guess I have a choice of who I could introduce here, because I also know Mark 
Musick well and I can vouch that he is one of the fine people involved in education in 
this country. I had the pleasure of serving on the National Assessment Governing Board 
with Mark, and there is nobody finer or more caring about education than he is. 



But Dr. Forgione and I actually worked together in the State of Delaware so it 
falls as my lot today to introduce him. Prior to the last four years during which he has 
been Commissioner of the National Center for Education Statistics, Pat served as the 
State Superintendent of Public Instruction for the State of Delaware from 1991 to 1995. 
That means he essentially ran education in the State of Delaware. In 1991, he served as 
Executive Director of the National Education Goals Panel so he had experience in that. 
And before that for four years he was a Division Director for Research in the State of 
Connecticut's Department of Education. 



He has, as you would imagine, won many awards and appointments including the 
Leadership Recognition award from the National Center for Education Statistics. He 
received his B.A. of Sacred Theology from St. Mary's Seminary and University, his 
Masters in Educational Administration from Loyola College, and his M.A. in Urban 
History and Ph.D. in Administration and Policy Analysis from Stanford University. He 
has more education than anybody could ever need. 



But the bottom line is he is this. He is extraordinarily energetic, he is obviously 
extraordinarily intelligent, he is extraordinarily apolitical, he is an individual who is 
devoted to what he does which is essentially trying to help children in education, and he 
has in my judgment, probably committed his life to that as much as anybody in the public 
arena in this country today. He is, I understand, not to be appointed again which I think 
is a tremendous loss for the country based on the personal experiences which I have had 
with him. 



I am also concerned about the integrity of the process. This actually started when 
questions were raised about some of the students who have learning disabilities not being 
properly included in the test statistics. Secondly, is a political question which I think is 
perfectly legitimate to raise before the panel and these questions do need to be raised. I 
can't imagine two better people to answer them. I am happy to introduce Dr. Forgione. 
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Chairman Hockstra. Not a bad beginning and introduction to Mr. Musick, the 
Chairman of the National Assessment Governing Board here in Washington and 
President of the Southern Regional Education Board in Atlanta. He has served as 
Chairman of the National Assessment Governing Board since October of 1996, as a 
member of the Executive Committee and Achievement Levels ofthat Committee. As 
Mr. Castle has indicated, he is also very well respected in the educational community. 

Welcome to both of you this morning. We will begin with you, Dr. Forgione. 



STATEMENT OF PASCAL D. FORGIONE , JR, Ph D., MS COMMISSIONER 
j OF EDUCATION STATISTICS, NATIONAL CENTER FOR EDUCA TION 
STATISTICS, OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT, 
US DEPARTMENT OF EDUCATION, WASHINGTON, DC 



Mr. Forgione. Good morning Mr. Chairman and.Members. I am Pat Foigione, the U.S. 
Commissioner of Education Statistics in the National Center for Education Statistics with 
.the. Department of Education. 



I was .originally going to summarize my answers to the 1 1 questions specified in 
the April 21 st Memorandum that are contained in my testimony. But given the recent 
events I believe it most important that I offer.some personal comments to set an 
appropriate context for this morning’s discussion. 



In light of the issues that have brought us here today I wish to clearly emphasize 
that over my 3-year tenure as U.S. Commissioner ofEducation Statistics, the integrity of 
NCES statistical processes have not been violated with respect to the identification of 
studies to be conducted, the design and implementation of those studies, and the analysis 
and content of our statistical reports.- However, there is one area that I believe needs 
-attention and that is the release of the Agency's statistical findings. This is one issue that 
has brought us here today. 



Over the last three years, I have become sensitized to and more acutely aware of 
the issue of safeguarding the independence of a federal statistical agency. My primary 
goal has been to protect the Nation’s data agenda. Critical to this is the release of results 
that are, and are perceived to be, objective and nonpartisan. There will be much 
discussion this morning on a variety of issues, but please do not be distracted from the 
heart of the matter, the integrity and independence of the statistical function in education. 



The landmark Report, Principles And Practices Of A Federal Statistical Agency 
produced by the National Academy of Sciences National Research Council, identified 
two essential prerequisites for the effective operation of a federal statistical agency. First, 
a clearly defined and well-accepted mission. Second, a strong measure of independence. 
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The Report states, "To be credible, a statistical agency must clearly be impartial. 
It must avoid even the appearances that its collection and recording of data might be 
manipulated by political purposes." 



Thus, it is in everyone's interest to have clear boundaries. The situation of 
ambiguity in releasing reports invariably leads to awkward situations that do not serve the 
Nation well. With the appetite for sound and unassailable education data at an all-time 
high, I believe it is all the more vital at this moment to give the National Center for 
Educational Statistics the institutional protection it needs to fulfill its statutory mission. 



I have already submitted my to you statement in response to your series of 
questions regarding the 1998 NAEP reading results. I am prepared to answer any 
questions following Mr. Musick's comments. 



Thank you very much. 



WRITTEN STATEMENT OF PASCAL D. FORGIONE, JR, Ph.D., US 
COMMISSIONER OF EDUCATION STATISTICS, NATIONAL CENTER FOR 
EDUCATION STATISTICS, OFFICE OF EDUCATIONAL RESEARCH AND 
IMPROVEMENT, US DEPARTMENT OF EDUCATION, WASHINGTON, DC 
SEE APPENDIX B 



Chairman Hoekstra. Thank you very much. Mr. Musick 



STATEMENT OF MARK D. MUSICK, CHAIRMAN, NATIONAL ASSESSMENT 
GOVERNING BOARD, WASHINGTON, DC 



Mr. Musick. Thank you, Mr. Chairman. I am Chairman of the National Assessment 
Governing Board, which sets policies for the National Assessment, and as you noted 
President of the Southern Regional Education Board. As you know, the National 
Assessment was created in 1988 by Congress. It is an independent, bipartisan, 
26-member group composed of state and local officials and yes. Congressman Castle was 
a valued member of that group. 
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The National Assessment sets policy and is the only program that provides sound 
comparable data, we would argue, on what American students know and can do. We 
have been doing this since 1 969. 



In many respects the National Assessment, as A1 Shanker called it, a treasure, has 
become the Nation’s most valuable and reliable report card on student achievement in our 
elementary and secondary schools. It is because of NAEP's role as an independent 
external measure that I believe the administration and the reporting of the National 
Assessment must be scrupulously honest, consistent and fair. 



The blue ribbon commission which recommended the National Assessment 
Governing Board's creation envisioned the Board as Congressman. Hoekstra said to buffer 
the National Assessment from what it called manipulation by any individual, level of 
government or special interest within the field of education. Now the blue ribbon 
commission referred to earlier was appointed by Secretary of Education Bennett, 
co-chaired by Governor Alexander, including Commissioner Forgione and the then First 
Lady of Arkansas. They recommended a structure that would supply needed checks and 
balances and separations of power for this important and sensitive enterprise. 



I would note that I have been appointed to the Board by Secretary Bennett, 
Secretary Alexander and Secretary Riley, and the current Vice Chair was appointed by 
Secretary Alexander and Secretary Riley. I say that only to emphasize the nonpartisan 
and nonpolitical. Congressman Mike Castle once said in 1994-95 during the 
reauthorization discussion and debate, "This may be the most nonpolitical group I have 
ever been involved with. Maybe you should be more political. We have tried to be 
nonpolitical, even without Congressman Castle's advice. His advice then was we needed 
to make what we were doing better known to Congress. 



The Board adopted its first policies about issuing results in 1990. Since that time, 
we have issued more than two dozen reports. There have only been two occasions in 
which we felt that our policies were not followed. Both of them have been described this 
morning, the first being in 1992 with President Bush, and the latest being in February 
with Vice President Gore. 



In both cases, the Chairman of this Board registered publicly, and we hope 
clearly, our concern that these actions were contrary to the Board’s policy and could in 
the long term harm the credibility of the National Assessment. You have copies of those 
letters. The Board met two weeks ago and we reviewed its policies after discussing the 
February release at its March meeting. We also had a release in March of the state 
reading results, and that followed completely as has nearly another two dozen releases the 
Board's approved plan. 



The Governing Board sets policy forNAEP under the law, but the Commissioner 
carries out that policy. This goes back to the 1 988 legislation where it set up this 
structure of checks and balances. The Commissioner implements the Board policies, but 
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as you know, he is an Officer of the Department of Education and falls under the 
Department's operational scheme. 



Now, you have my statement about the release. I think the important thing from 
the Board’s standpoint is that two weeks ago we revised our policy slightly and stated that 
the initial release of NAEP data shall be independent and apolitical, and separated clearly 
from other programs and policies. We have added that the official press statement or 
release announcing NAEP results shall be issued by the National Center for Education 
Statistics, rather than by any other Department, any other part of the Department of 
Education. 



With regard to the issue of exclusion of NAEP students, the Commissioner has 
had a preliminary report which he presented to the National Assessment Governing 
Board two weeks ago, and he is prepared as I am, to comment on that issue as well this 
morning. But I will stop with my initial comments on the release of NAEP results. 



WRITTEN STATEMENT OF MARK D. MUSICK, CHAIRMAN, NATIONAL 
ASSESSMENT GOVERNING BOARD, WASHINGTON, DC - SEE APPENDIX C 



Chairman Hoekstra. Thank you both very much for your testimony. 

My colleague has described what happened as just a social faux pas by the Vice 
President, but throughout both of your statements I sense a real commitment and a real 
urgency to maintaining the integrity and the independence of the process. Is that correct? 



Mr. Forgione. Yes. 



Mr. Musick. Yes. 



Chairman Hoekstra. For what reason is that? 



Mr. Forgione. As I tried to state it is very important. I think the fact that we are here 
today with the interest of so many actors that perception becomes reality and therefore 
how we proceed is very much at stake in the belief in the data. I do believe this was 
inadvertent, but I think it points out the need to have more clear boundaries so that this 
will not be an issue in the future. 
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Chairman Hoekstra. And you strive for this in all aspects of your work? So it not only 
the release of the data, it is also independence as you structure the tests and those types of 
things. Is that correct? 

Mr. Musick. I would certainly agree with that. We, the Board and I think the 
Commissioner would probably agree, can be accused of being purists when it comes to 
NAEP data. We do believe that ultimately the integrity of this data when you get to the 
core of it is what you have. 

We know the data will be interpreted. We expect that. However, we also believe 
the data will drive the debate. We know there will be debates about this data. Our 
concern is that the facts be presented and released in a nonpartisan, nonpolitical way, and 
then let the debate and the interpretations begin. 

Chairman Hoekstra. I am trying to think. Who was it, Joe Friday, "Nothing but the 
facts." That is kind of your job. 

Mr. Forgione. The Board can have more in expression. It is the Commissioner who is 
really Jack Friday in the sense of facts only. 

Chairman Hoekstra. Joe Friday. 

Mr. Forgione. The Board can make comments about what this means, and they do it in a 
balanced way. The Commissioner certainly has no interest or perspective on what the 
data means. 

Chairman Hoekstra. I think this is why this issue is important to me and why I think it 
should be important to Mr. Roemer. As you described it, this process in the work that 
you do begins the debate, it does not end the debate. Is that correct? 



Mr. Musick. Yes. Yes, sir, I believe that is right. 



Mr. Forgione. Yes. 

Chairman Hoekstra. If part of the debate is whether it was used for political reasons or 
it is driven by political motivations or these types of things, it really doesn't help the 
process at all. That is why you so jealously guard your independence from political 
interests, correct? 
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Mr. Forgione. Yes. 



Mr. Musick. Yes. 



Chairman Hoekstra. Mr. Musick, regarding your restatement of how the results are 
supposed to be released in the future, does it say that these will now be released only by 
the Commissioner of Education Statistics, or does it just say independent from other 
agencies within the Department of Education? 



Mr. Musick. The policy reads that the official press statement or release announcing the 
NAEP results shall be issued by the National Center for Education Statistics rather than 
by any other part of the Department of Education. 



As you know, all of the Secretaries since the National Assessment Governing 
Board has been formed, have participated in the releases of data periodically. But what 
we are saying is the Commissioner speaks first, the Commissioner puts the data on the 
table first, and the Commissioner presents the facts first. Then any interpretation by the 
National Assessment Governing Board, the Secretary or anyone else follows either that 
presentation the next day, in coming months, or in coming years in fact. 



Chairman Hoekstra. Mr. Forgione, for the release we are talking about that is the 
agenda that you propose, correct? 



Mr. Forgione. That the Governing Board had approved and that I brought to the 
Department, yes. 

Chairman Hoekstra Did you have discussions with the Secretary. and the Vice President 
about that? 



Mr. Forgione. Yes. Towards the end of this process interest was shown which often 
happens, and at that point we had a conversation to try to allow for that after the 
Commission and Governing Board would present the data. Really, there is much 
flexibility. It can take place in 5 minutes or that afternoon, and that was the intention that 
I believe was communicated and that we held to. 



Chairman Hoekstra. My time has expired. We are going to break. We have got a vote 
on the journal on the floor. We should be back in about 10 or 12 minutes. At that point 
in time we will move to Mr. Roemer. 



[Recess.] 
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Chairman Hoekstra. The Subcommittee will reconvene. Mr. Roemer. 



Mr. Roemer. Thank you, Mr. Chairman. Mr. Forgione, let me ask you a couple of very 
simple, easy questions. I am also concerned about the independence and the 
nonpartisanship and the honesty and consistency and fairness of this process, so let's get 
right to it. Is there anything that has politicized the collection in this process? 



Mr. Forgione. No. 



Mr. Roemer. Thank you. Is there anything that has politicized the contributions to this 
process? 



Mr. Forgione. No. 



Mr. Roemer. Is there anything that has politicized the analysis in this process? 



Mr. Forgione. No. 



Mr. Roemer. So the whole question comes to the release. There is no question of the 
integrity of the process, the compilation of data, the analysis of data. Everything is by the 
book. It all comes to the release that we are talking about here today. 



Mr. Forgione. Yes. 



Mr. Roemer. Okay, that is helpful. Now, on page 4 of your testimony you say the 
release, according to your guidelines, "This can take the form of a publication, a data file 
and/or an electronic release." 



Now, this was released an hour before the press conference, electronically. Is that 
correct? 



Mr. Forgione. Yes, it is. 



Mr. Roemer. So according to your guidelines, even the release of this material was by 
the book? 



Mr. Forgione. There is a subtle but important difference here. When I was asked the 
question I answered it honestly, that in my view it does affect the perception of 
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objectivity in this case. But I do believe it was probably inadvertent and that is an 
important perception. I think a lot of the consternation we are facing, and as you said the 
distractions from looking at the data, are because of that distraction. 



Mr. Roemer. Go to page 5 of your testimony and get right to that. You say on page 5, 
"The Commissioner electronically released the national NAEP data on the NCES web 
site at 10 a.m. that morning." Then you go on to say basically what you just said to me. 
"However, the fact that someone other than the Commissioner was the first one to 
announce these findings publicly at the press conference created a perception." 



What we are here to clarify and spent all week with staff time and Member time 
and so forth is what you have now announced in your new guidelines, with an addendum, 
that now nobody in politics can release this data in the future. How long ago did you clear 
this up? When did you add that? 



Mr. Forgione. Two weeks ago, I believe. 



Mr. Roemer. Two weeks ago. Before we had this Hearing, you had clarified your rules 
to say whether it is President Bush, Vice President Gore or whether it is President Gore 
or President Bush in the future. We now have clarification on exactly who can release 
this data, is that correct? 



Mr. Forgione. Yes. 



Mr. Roemer. So we think we have solved the problem, is that correct? 



Mr. Forgione. Yes. 



Mr. Roemer. So why are we here? 



Mr. Forgione. When one is asked to come and answer questions about an important 
issue, one does that. 



Mr. Roemer. I am glad you showed up. You would probably have been in trouble Mr. 
Forgione, I am being facetious, if you said no. 1 think you have answered all my 
questions very clearly about the integrity and the honesty and the fairness and the 
independence of the process. We have attempted to clear that up. We have attempted to 
make sure that in the future there is not even an iota of perception problem here with who 
releases this data in the future, whether it is electronically an hour ahead of time. 
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Now we have clarified the rules as of 2 weeks ago. You did it independently. 
Somebody in politics cannot release this in the future. 

You do say, or at least it has been said by your predecessor, Mr. Musick 
Mr. Boyd said in his letter where he is complaining and surprised and disturbed about 
President Bush releasing this prematurely, a day early you say, "Of course, we are 
pleased by the President's interest in NAEP and realize his attention adds greatly to 
public awareness of NAEP results." 



My final question would be now that we have clarified this and seem to be 
moving in a direction where there wouldn't be a perception problem in the future, how do 
we balance the line between making sure that this whole process is independent, fair, 
nonpartisan with showing that there should be after it is released some political interest in 
this? Mr. Musick? 



Mr. Musick. Yes. We and others have often cited some examples, the Bureau of Labor 
Statistics gets cited quite often, of releasing important economic statistics. Then those 
statistics are commented on elsewhere or commented on later following that official 
release. Without stretching the analogy, I think that is at least in the general direction that 
we see the National Assessment operating. 



Obviously others had said at the time that reading is so important in America, 
how can it be a problem when lots of people want to talk about reading? And that is a 
persuasive argument. We, and again I don’t apologize for being purists in this, believe 
Joe Friday should put the facts on the table. There should be, as you pointed out, no 
question about the integrity of those facts. The discussion and interpretation should take 
place all across America, not just in press rooms in Washington, D.C. 



Mr. Roemer. Thank you, Mr. Musick. 



Mr. Forgione. If I could add one comment, the Governing Board assists the 
Commissioner in the release of the National Assessment data. We have a much wider 
portfolio of education data. I would ask you and the Chairman to consider as you are 
looking at the reauthorization of my agency presently, to think about putting protections 
that would institutionalize this relationship so that the boundaries are very clear and it 
will not lead to these awkward solutions. 



Mr. Roemer. Thank you. Thank you, Mr. Chairman. 



Chairman Hoekstra. Thanks, Mr. Forgione, for those comments. I think that one of 
the things we are looking at here in the Subcommittee as we consider reauthorization is 
how do we build those protections into the process to ensure that this is a fully 
independent agency. It may require legislative changes rather than just some internal 
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rewriting of the rules and those types of things. Mr. Castle. 



Mr. Castle. Thank you, Mr. Chairman. I came here and was interested in this Hearing 
because of the problems with the test itself. Now 1 am concerned about the independence 
of the agencies and also about the Vice President's role and 5 minutes isn't going to do it. 



I would like to get to the bottom of all of this. Maybe I should ask Dr. Forgione. 
What is your judgment with regard to the National Center for Education Statistics on 
institutional protection? As I understand it, it is strictly under the guidance and control of 
the Department of Education. I don't mean in a political sense, it is just they control who 
works there and who doesn't work there. They control most of the activities of the 
organization. 



Should it be, and you are not to be reappointed as I understand it, more 
independent? If I have time I am going to ask Mr. Musick should NAGB and NAEP be 
totally independent? What about the agency you work for? Should it be more 
independent from the Department of Education? Would that help with some of the 
depoliticizing of it? 



Mr. Forgione. To provide the Commissioner with the authority in the area of personnel, 
budget and the release of reports would be a substantial enhancement to that integrity. 
Obviously you can work it out and I have had a terrific relationship with the Department. 
But it does take energy and roadblocks come up and priorities can bump your priorities. 



Other statistical agencies do have this protection. Look at the legislation for the 
Energy Statistics Organization and Bureau of Labor Statistics. I don't think anyone would 
fault that especially at this moment with the prominence of education data. You know, 
during the eighties no one was listening to the data. It didn't have the same capital. These 
are so important now I think you need to think about this new context. 



Mr. Castle. You may not have done this Dr. Forgione in your statement but I have a 
whole notebook filled with questions. I would love to have in writing your thoughts 
about how the Agency that you are heading now should.be handled. 



SUBMITTED FOR THE RECORD, LETTER FROM PASCAL D. FORGIONE, JR, 
Ph.D. TO PETE HOEKSTRA, CHAIRMAN, SUBCOMMITTEE ON OVERSIGHT 
AND INVESTIGTIONS, COMMITTEE ON EDUCATION AND THE WORKFORCE, 
JUNE 14, 1999, PAGE 3, ITEM #13 - SEE APPENDIX G 



Mr. Castle. Mark, could we ask you the question on NAGB and NAEP? I have often 
thought that maybe they should be completely independent, and also your relationship 
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with the National Center for Education Statistics. That was always a little bit of a 
troubled area when I was on the NAGB board. What are your thoughts about the 
independence and relationships of these organizations, as rapidly as you can? 

Mr. Musick. I will make three points. When the Alexander James group considered this 
for more than a year in 1987 and came up with the proposal that Congress adopted in 
1988, it envisioned a structure with shared power, a board, a contractor in the Federal 
Government. 



A year ago That Congress gave the Governing Board the exclusive authority for 
something called the Voluntary National Test Proposal. Now we have exclusive authority 
for something that does not yet exist and we have shared ambiguity for the National 
Assessment program. So this sharing arrangement I would argue has worked but it does 
create ambiguity. The Board is trying to deal with the difference between exclusive 
authority and shared ambiguity, Mr. Castle, and I would have to think more about that. 

Finally, today the relationship with the National Center of Statistics and the 
Governing Board, I would argue, has never been stronger. Commissioner Forgione's 
relationship with the Board, his openness, the way that we worked through the redesign 
for the National Assessment showed that he did not try to play things real close to the 
vest. 

Mr. Castle. That may be institutional, may it not? Maybe I am referring to a different 
time when I said it was a little more fragmented. Should we institutionally change it to 
make sure it is always like that or the proper distance is there? 

Mr. Musick. I think you can argue that the reason it is more personal today than 
institutional is that it reflects a leadership style. 

Mr. Castle. I would appreciate hearing any thoughts you have on that. Let me ask you, 
and I am going to run out of time sooner than I would like to, on this particular round of 
tests there were some serious questions raised, at least in certain states, about certain 
students who might have achieved at lower levels. Learning disabled programs for lack ol 
a more inclusive expression, were not given the test in the same percentage as they might 
have been in other states or other years. That really bothered me. That bothered me as 
much as anything. 

I would like your comments on that. 1 have spoken to Dr. Forgione about this, 
and I don’t consider it to be de minimis. I consider it to be significant. If you are going 
to give these tests, it is absolutely imperative they be fairly given and depoliticized. If 
there was a problem, we need to get to that and make sure that that is being eradicated. 
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Mr. Forgione. I can say to you unequivocally, the administration of the National 
Assessment at the national.level and in the 43 states that took grade 4 and the 40 that took 
grade 8 had no incidence of any abrogation of administration. 



What is happening is in the state contexts they are serving more kids in Special 
Ed, and those individual plans are identifying and maybe not yet perhaps giving the 
instructions when you can take the test and not take it. That is leading to variation. But I 
can also say to you the Study we just completed says this had modest impact on the 
results. I know the numbers look big. 



Mr. Castle. But a little bit of that, with all due respect, is educational or understandable 
language. Lwas going to say gobbled ygook The real bottom line is there are some 
differences in some of the people taking the test. You may argue it is not significant, but 
because of what you just went through there are some differences in who is taking the test 
and who isn't. Maybe we are not comparing the same oranges to the same oranges every 
year to year, or 2 years to 2 years. 



Mr. Forgione. All you can do is go into a state and say all children should be tested. 
Give me your fourth grade list. We pick the 30 kids. Then the principal can only say Pat 
and Mark can't take the test because his IAP says so. We are driven by that. There is not 
uniformity throughout the country as to where those things happen. That is the only 
difference. 



So I think we have uniformity as best we can. We are going to tighten up with the 
Board's help. If there is variation in the future, we may not produce your result because 
we may believe they lead to an unfairness. 



But you are absolutely correct, the Governing Board is committed to a perception 
and reality of fairness. At this moment I can tell you, the 1996 mathematics test was not 
consequential in this matter, and I can say to you it is only modest here as you can read in 
my statement. 



Chairman Hoekstra. Mr. Scott. 



Mr. Scott. Thank you, Mr. Chairman. First of all a very brief question in terms of which 
is worse. Is it worse to participate in an announcement after the data is on the Internet, 
after the results are on the Internet, or is it worse to make announcements at political 
fund-raisers the night before the report is released? Is there any question that announcing 
at a political fund-raiser the night before is worse? 



Mr. Musick. Both of those were... 
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Mr. Scott. There is no question that announcing it at a political fund-raiser the night 
before the report is issued is worse. Is that a fair statement? 

Mr. Forgione. I would agree they are both inappropriate. 

Mr. Scott. You would agree that a political fund-raiser is worse, if anybody can get the 
information off the Internet after it is already on there? You have problems saying that it 
is worse to make the announcement at a political fund-raiser in terms of which more 
politicizes the report? If you have a problem saying it is worse, just say you have a 
problem saying it is worse. It is a simple question. 

Mr. Musick. Congressman Scott, the two occasions I would argue were both in 
violation of the policy. The things that were said at the two occasions differed quite 
dramatically in terms of the breadth and depth of the comments. 

Mr. Scott. So you have problems saying that announcing before the results have been 
made public at a political fund-raiser is worse than making, participating, in an 
announcement after the report has been put on the Internet. You have a problem saying it 
is worse? 



Mr. Forgione. I don’t think we addressed the fundamental question, which is the 
perception of integrity and the belief in the data, and one must address that. The degree of 
difference I don’t think is really at issue. What is at issue is the public’s confidence that 
the results are correct. 



Mr. Scott. I thought it was a simple question. Obviously it wasn’t. You have a problem 
saying that politicizing it, bringing it up at a political fund-raiser before it is announced, 
isn't worse than commenting on it after it has already been made public. I just say that 
because the other comments have to be taken in the context of the way you have 
answered that question. 

Let me ask about the results themselves. Have the results been validated in terms 
of being meaningful? Do students who do well on the tests get better jobs, get better 
grades, are more likely to get into or perform well in college? 

Mr. Forgione. At this point, there are not validity studies that are associated with the 
performance levels of the Governing Board. In other words, if you are basic, proficient, 
advanced, what are the consequences in the future for that? The Governing Board is 
using a judgmental process that is equal to what the states do, that says have you met the 
standard or not. They haven’t gotten into what we call the "consequential validity" of it. 
It is more the base validity and that the test represents the constructs that the public has 
said are important, and the tests measure those. 
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Mr. Scott. The concern obviously is that some of these standardized tests have a racial 
bias, and if they are not validated then that would obviously cause concern. Is there any 
attempt to try to get validation? 



Mr. Forgione. Well, we do bias analysis. I wouldn't call that validation but I can see in 
a common sense that you can use that. There is strict bias analysis done on every item to 
ensure that it doesn't have a difference impact on different categories of students because 
we want to use an item that would be fair and reliable. Therefore, that is strictly done. 
The National Assessment is probably the premier assessment in this country, if not the 
world, given the investment we make in it. 



Mr. Musick. The kind of validity question I think you are asking, Mr. Scott, for the 
National Assessment, gets, as Congressman Castle would testify, a little complicated, 
because individual students don’t actually take a National Assessment of course. They 
take a slice of the National Assessment. It is a survey examination. 



Mr. Scott. My time is about to run out now. I want to get in one other question. Do you 
see certain variables in terms of good results? Do you see better results with smaller 
class sizes, better teachers, higher paid teachers? Can we learn anything through the 
assessment in terms of what we ought to be focusing on to improve education? 

Mr. Forgione. The nature of the National Assessment is a survey assessment, not a 
testing program. We can tell you how the Nation is doing over time or a fourth grade 
class in 1992, 1994 and 1998. It doesn't have the ability to look at evaluative effects. It 
is not designed as a longitudinal study, it is cross-sectional. So you can't make those 
kinds of conclusions. 



Every now and then we can see a correlation and we try to do secondary analysis. 
But the design is not meant to ask if Title I is working or is this working. We can tell you 
if poor kids are learning, but we don't know what program those treatments come from. 

It is an indicator, not a program evaluation tool. 



Mr. Scott. Mr. Chairman, I don't have time to follow through on this, but it seems to me 
that if you are doing a survey, you ought to have enough statistical data to draw certain 
conclusions, and if we are going to be giving the tests we ought to be able to cull out 
some of this information. I would like to follow through on that at a later date. 



Chairman Hoekstra. Mr. Fletcher. 



Mr. Fletcher. Thank you, Mr. Chairman. I appreciate you conducting this Hearing and I 
appreciate both of your testimonies. Let me start with making a statement. 
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You know, the object of this was really to depoliticize the educational assessment, 
for us to be able to really tell how are we doing, and how the programs we are 
implementing are doing. It is not for individual students to predict if they are going to be 
doing well. So we need to keep focused on that. 



I want to look at how it has been politicized. When I look at the comments that 
Vice President Gore made, he said, "I am pleased today to be able to release the results." 
Clearly this was a premeditated, intentional effort to do something that was against the 
regulations and the intent of what we have established here. 



He says, "Well, today I am very proud to be able to report to you new evidence 
that our efforts are beginning to pay off. For the very first time, reading scores have 
improved for each of the three grades measured by the National Assessment of 
Educational Progress: 4th grade, 8th grade and 12th grade. This is great progress and we 
are proud to report it." 



I looked at the results, and the fourth grade achievement levels showed no 
statistical significant change. Fourth grade scale score net gain is constant, 8th grade 
scores do represent progress in short and long term and 12th grade scores overall showed 
no change from 1 992 to 1 998. 



It looks to me as if not only did he go out of his way to report something that he 
really shouldn't have done and to use it for political gain, but he exaggerated extremely 
the results of the test for political gain. There is certainly apples and oranges when you 
look at a side remark that was made by someone who is not in public office anymore and 
is not going to be there in the future. 



Let me ask a few questions, especially that relate to Kentucky. First off, I want to 
say I understand there are about four states that registered large increases in the exclusion 
rates from 1994 to 1 998, and Kentucky was one of those. Also the adjusted scores 
appeared smaller for all four states, but the results remained statistically significant for all 
but Kentucky. I just want to ask a question about that. As I understand, Mr. Musick, 
would you please explain why the NAEP exclusion policy changed in 1 996? 



Mr. Musick. In part because there are lots of things changing across America. I think 
this gets to a question of Congressman Scott. We are seeing a time of state efforts to 
expand services for disabled students and to test them in Kentucky. The KATS program 
tests them under nonstandard conditions with what is called accommodations, giving 
students more time and helping them take the test. 



So the states all across America are trying to include more students in their testing 
programs. NAEP has had a longstanding tradition of trying to include more students. So 
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the reason we changed that in 1996 was to try to take into account what is going on 
across America while, and this is a big "while”, maintaining our trends and our ties to 
what went on 2 years and 4 years and 6 years before. 



Mr. Fletcher. Do you think that could have resulted in any way in some elevated scores 
or looked like there was progress because of changes in the sampling? 



Mr. Musick. Well, the Commissioner is better prepared than I to comment on that. I 
will simply say we will be releasing written results in a few months, a few weeks really. 
We will be looking at the kinds of questions you are asking, Congressman Fletcher, the 
kinds of questions our analysis has done. Before we changed the exclusion- inclusion 
rules, we did a year of pilot testing. We looked at it both ways. We used tests with no 
accommodations and tests with accommodations. 



Mr. Fletcher. Let me interrupt. I have one more question. How do you know that states 
interpret inclusion rules in the same way? Kentucky has been used as an example of how 
the interpretation of this policy.can really change scores. How do.you ensure uniformity? 



Mr. Forgione. We administer these tests at the national level. We go out and do it in 400 
schools. In the states, we train independent individuals who go into the schools and 
oversee it. So the states are not controlling this, they are using the common criteria and 
applying it. So we have the stability. 



We have also looked over time if the old criteria and the new criteria has changed 
the national measure? There is an old maxim: Don't change the measure if you want to 
measure change. So that was the tension we were in. We did a study saying that the new 
criteria and the old in mathematics gave us the same distribution, so statistically we could 
use the new criteria, and that is why the Board went to that. 



Mr. Fletcher. My- time is up. I appreciate your answering the questions. I would say in 
closing I really think some of .the politicization of this shows the dangers of manipulation 
and the political abuse that would take place if we really established national testing. 



Thank you. 



Chairman Hoekstra. Mr. Ford. 



Mr. Ford. Thank you, Mr. Chairman. 1 am pleased in some ways that I am hearing a lot 
of concern from my colleagues on that side of the aisle. I know there is genuine concern 
for how we can improve the quality of education in America. I am a little dumbfounded 
as to how we could be remotely upset that anyone could announce the fact that we are 
seeing some progress and movement in the right direction as relates to kids' reading. 
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I am from the state of Tennessee not far from Kentucky and my friend Mr. 
Fletcher and the others on that side. It amazes me that we would really call a hearing to 
criticize the Vice President for announcing the fact that we are moving forward 
particularly after our witnesses, Mr. Scott and I are a little offended, whom we are glad 
are here, couldn't answer the obvious. These findings that the Vice President announced 
were already released or were already electronically released. 



President Bush, who I had great respect for and I hope his son remains Governor 
of Texas, Mr. Fletcher, made the comments at a fund-raiser prior to the release of these 
findings. The Vice President, it is my understanding, made this announcement after the 
release at what could be considered a political event, and we are all politicians so 
anything we do is for political gains in many ways. So let us not be that naive and 
certainly ignore political realities. We are not going to all try to make gains. 



But it just amazes me that we would actually hold a hearing on something so 
unimportant, when we have had to fight and fight and fight to convince my colleagues on 
the other side of the aisle that we ought to maybe explore what happened in Paducah, 
Pearl, Littleton and Conyers. We pray for those families and communities as they try to 
heal. 



I would just have a few questions for the Commissioner. Commissioner Forgione 
forgive me, I am from Memphis and we have problems pronouncing those last names. I 
hope I am pronouncing your last name correctly. During your tenure at NCES has 
anyone ever abridged your ability to design or implement any study or any NAEP 
studies? 



Mr. Forgione. No. 



Mr. Ford. Did the electronic release of the NAEP results at 10 a.m. on February 10th 
fulfill your responsibility as Commissioner to release the data? 



Mr. Forgione. It fulfilled it in one sense, but normally when we have a press event, the 
press event is seen as the more visible and therefore can set a context that one has to take 
into account. Electronically, typically, is done when we don't have a press event and get 
it out to America. Sometimes I always electronically release it, because I want the world 
to see our information. 



Mr. Ford. So you wouldn't be surprised if you released something for the world to see 
and a little elementary school in my District which had access to the Internet saw it and 
decided to call the media and said hey, we see that reading scores are improving for 
students at our public schools throughout this District. You wouldn't necessarily be 
offended at that fact, would you? 
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Mr. Forgione. No, we would compliment them. 



Mr. Ford. I have only been here a term and a half, but it sounds like to me we have folks 
crying over sour grapes. Maybe if somebody down in Texas had had an opportunity to 
announce these results a little sooner, we might not be as upset as we are now and might 
not have been provoked to call a Hearing. 



The Vice President didn't invent the Internet, I think he did. Some folks don't 
believe he invented the Internet, but I think he did. He just happened to find this 
information a little quicker than other folks and happened to have a speech a little later 
that day announcing the fact that the President and he are. committed to redoubling their 
effort to educate kids. I am trying not to be partisan here. He just happened to announce 
the fact after it had been. released across the Internet that we are seeing some 
improvements. 



I know my friend Mr. Hoekstra would have to be pleased, as he turns on my light, 
that we are making progress on these fronts. 



I hated to have dominated my .time and not allowed Mr. Musick to have a long 
period of time to respond. I know the question has been asked before, but I just want it in 
the record one more time. I know Mr. Scott tried to get at this, but would you 
characterize President Bush's action of announcing the 1992 results at a fund-raiser. 

I think l am quoting you sir, and if I am not correct me, as an event that was not 
consistent with the principle of an independent, nonpartisan release of National 
Assessment data? I think you might have made that comment sir, if I am correct, as 
related to Mr. Gore. 



Mr. Bush’s comment, and Tthink he had every right to make the announcement 
even if he decided to try to use it for political gain, which it is kind of hard to believe. 
We are shocked'that some of us up here would make statements for political gain. 



Mr. Musick, did you make that comment regarding Mr. Gore? Would you say that 
that probably would apply to the. distinguished President? 



Mr. Musick. Mr. Boyd, my predecessor said he was surprised and disturbed by Mr. 
Bush's - actions. I would say that my words would apply equally to both occasions, yes 
sir. 



Mr. Ford. Mr. Chairman, I appreciate the time, but again I think what it boils down to is 
that the Vice President found out this information a little quicker than other folks and 
made the announcement, and now we are going to accuse him of politicizing the fact that 
our kids are reading a little better. I will be continuing to politicize those things and hope 
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that our kids will continue to read better and add and subtract and multiply and divide a 
little bit better, too. 



I thank you for letting me go over my time, Mr. Chairman. I thank the witnesses 
for coming. 



Chairman Hoekstra. I thank Mr. Ford for his questions. I just add that I can now 
understand if you believe the Vice President created the Internet, why you also believe 
his characterization of the test results. Mr. Schaffer 



Mr. Ford. Thank you, Mr. Chairman. 



Mr. Schaffer. Thank you, Mr. Chairman. I am not sure which panelist is the most 
appropriate to comment, but NAEP used a time frame in assessing the overall 
improvement of students from 1992 to 1998. Am I correct in that? 



Mr. Musick. Correct. 



Mr. Schaffer. I think the bigger issue is not so much whether some politician decided to 
exploit the results of the data for whatever political advantage may be secured, but how 
that data was released and under what pretense as well. The time frame used by the Vice 
President was not the same as yours. Am I correct? 



Mr. Musick. The figures were for 1992, 1994 and 1998. I think the Vice President may 
have focused on 1994 and 1998. 



Mr. Schaffer. As opposed to 1992 and 1998. When focusing on the 1994 to 1998, and 
let me just quote what the President said, he said, ’’Well, today I am very proud to be able 
to report to you new evidence that our efforts are beginning to pay off. For the very first 
time, reading scores have improved for each of the three grades measured by the National 
Assessment of Educational Progress: 4th grade, 8th grade and 12th grade." And then 
with emphasis, "This is great progress and we are proud to report it." 

In your presentation of the data, however, did you come to the same conclusion? 



Mr. Forgione. When I presented the data as I do in each and every case, I go back to 
the base year and talk first about the base year, 1992 versus 1998, in which there was no 
change at grade 4 and no change at grade 12 and grade 8 did have change. Then I will 
take an intermediate period, because in this case there was only one, in which there were 
gains over the short-term. 
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We have a quadratic where we lost ground, came back, and that is why it is 
equal at 4 and 12, but 8th grade was our success. So the Commissioner and the Board 
presented the full spectrum. The Vice President did focus on that intermediate period. 



Mr. Schaffer. Just from the perspective of the value of the NAEP scores and what they 
tell the American people about the performance of their children, is there some difference 
just in the intellectually honest application of the findings when considering these time 
frames? In other words, what would be more honest in your opinion, the 1992 to 1998 
comparison, or just the selective portion of 1994 to 1998? 



Mr. Forgione. They both have validity, and one would have to put them in context. 
NAEP is the goal standard. It is done independently and it is done with integrity. That is 
why it is so important to release them in a way that people can interpret for themselves. 
That is why the Commissioner, I believe the Chairman said, is Jack Webb. We know the 
facts only, and let you decide what they mean. 



But I don’t want to underestimate that, because that is so important. Let's not just 
get distracted into who struck John, but what is at the heart of this. It is the integrity and 
the independence of a statistical agency and the perception that that may be a bridge, and 
no one wants that. Sol hope you will address that as you go forward. 



Mr. Schaffer. Mr. Chairman, do you need any of my time? 



Chairman Hoekstra. I don't need your time, but I will take it. 



Mr. Schaffer. I yield my time to the Chairman. 



Chairman Hoekstra. I would like to insert for the record the National Assessment 
Governing Board Policy Statement. I believe this is from 1990. Without objection? 



SUBMITTED FOR THE RECORD, POLICY STATEMENT ON REPORTING AND 
DISSEMINATION OF NATIONAL ASSESSMENT RESULTS, NATIONAL 
ASSESSMENT OF EDUCATIONAL PROGRESS, NATIONAL CENTER FOR 
EDUCATION STATISTICS, ATLANTA, GA, NOVEMBER 17, 1990 
SEE APPENDIX D 



Chairman Hoekstra. I think your intent at the press conference as you talked about the 
data and your agenda differed. Your proposed agenda differed from the actual agenda in 
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what way? What did you propose and what actually happened? 

Mr. Forgione. The order of the presenters, in accordance with the NAGB policy, is that 
the Commissioner should release all the data and put the full spectrum out, then allow 
commentary at some point afterwards, immediately or later. In this case the Governing 
Board, the Secretary and the teacher would have joined us. 

Chairman Hoekstra. So the National Assessment Governing Board policy is the data is 
presented first, after which there may be time for questions on the data or somebody may 
offer a policy statement. That is how you jealously guard it. What actually happened at 
the presentation? 



Mr. Musick. At the presentation as has been established, the data was technically 
released on the Internet at 10 a.m. At 1 1 a.m. a press conference began with the Secretary 
introducing an outstanding teacher from Fairfax, Virginia. She introduced the Vice 
President. The Vice President released, his term, "released" the results. The Vice 
President left and Commissioner Forgione presented the results, and then I was on the 
program commenting on those results. Then there was a question and answer period with 
the press and persons in the audience. 



Chairman Hoekstra. So the Vice President gave a rather sterile presentation of these 
are the numbers. Nothing but the facts? Is that what happened? 



Mr. Musick. I think you have heard some of the testimony or some of the results. 



Chairman Hoekstra. All right. I will get my 5 minutes again in 5 minutes. Mr. Kind. 



Mr. Kind. Thank you, Mr. Chairman. I want to thank the witnesses for coming even 
under the environment in which you are being asked to testify. I for one just want to 
echo what our Ranking Member said in regards to this hearing and what the purpose 
really should be about. That should be figuring out ways that this Committee can work 
together in a bipartisan fashion to try to improve the state of education in this country. 

Quite frankly, I think it is an incredible waste of time to hold a hearing of this 
nature with this kind of content and to waste staff time for a few days talking about the 
release of information which to my understanding was published over the Internet for 
public consumption one hour after it was published, and then try to make some political 
hay about it. 

If the next year and half in this Congress, Mr. Chairman, is going to be nitpicking 
and criticizing the Vice President of the United States for political purposes, we are in for 
a very long time, and I fear that we are not going to get much accomplished if anything. 
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I would think that when it comes to concerns about our children and the 
education that we are supposed to be providing in this country, that we can rise above this 
on occasion and keep our eye on the ball rather than wasting a whole lot of people's time. 



I mean it is my understanding that after an hour of this information being 
publicized, the Vice President made some obscure comments in regards to the test results. 
This is a very fine line between political use and a policy discussion. There is no question 
about that. But this certainly cannot compare with a sitting President announcing the 
results of NAEP test scores a day before they are even published at a private school, and 
then shortly thereafter at a fund-raising event with the Vice President of the United States 
in attendance. So, let's be realistic about what this is all about. 



What concerns me more is the fact that we have got what appears to be at least 
some manipulation of NAEP test results going on right now. I would like to hear your 
comments in that regard, because in all honesty, a lot of school districts are making some 
policy changes in order to accommodate and to include students with disabilities. I 
would like to know what is being done from your point of view in order to deal with 
these policy changes so that we don't have a state-by- state potential manipulation of 
future test scores and we can get some accurate and quantifiable data that are reliable in 
the future. If anyone would like to comment. 



Mr. Forgione. Please, in my statement we do address that issue. I was concerned after 
the initial release of the state data. People raised the question. The number of identified 
Special Ed kids has gone up. We can understand that. America is funding more, so you 
would expect more children to be identified. But is exclusion happening? 



We did a study, Mr. Kind, to make sure that we could identify the impact on it. I 
am pleased to say that despite the look of a large number, we can say to you it is only a 
modest impact. Only 14 of 36 states that were studied in 1992, 1994, and 1998 had a 
significant increase in the number of Special Ed kids. Therefore, this problem is a 
modest one at this point. We do recognize that we have got to be vigilant about it and 
make sure we put in place procedures that will not allow results to be reported that are 
not fair. 

The answer to the question is there was no manipulation. We administered the 
tests independently. We picked the schools in every state. We picked the roster of 
fourth graders and we drew 30. Only at that point could the principal or administrator 
say Mark and Pat should not be tested because their IEP says so. So that is a fixed 
document that continues to be manipulated. It is there. 



Now, why do different numbers pop up in different states? That is a contextual 
issue relating to the services and the funding and their priorities. We tried to have a level 
playing field. Certainly the state over time hopefully would be level. 
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We are vigilant about this. I can say to you with confidence, these results hold. It 
did not apply to the Math results released in 1996, but it certainly is an issue that the 
NAEP and the state governments with their state testing programs need to be vigilant 
about. 



Mr. Kind. In those states where you feel there may have been some exclusion rates 
affecting test scores, were there any other factors that may have attributed to some of the 
different test scores in those states? 



Mr. Forgione. We only had time to look at this one factor: What impact did the 
increase in exclusion between 1994 and 1998 that went from 4 percent to 8 percent have 
on your performance? If you had growth? Would raising the score up impact? We were 
very harsh. We took out the bottom kids and raised the 1994 scores, and then asked is 
there still a statistically significant increase or are you still statistically the same? No one 
went backwards, and 33 of the 36 states showed that they maintained their growth. 



So in fact I can say to you with confidence that even this rigorous model didn't 
have the impact. We are doing some reasonable models, because Special Ed kids are not 
all at the bottom. Some ADD kids. are terrific performers. We will have those studies. 
Two of the 3 States that did not show significance at this point, may in fact with the 
■reasonable model be able to be certified. But the results will not change. This is a 
hypothetical study, "what if." 



The results are what they are. You could ask the same question if you had more 
migration or you had more poverty. We take the data in NAEP and report it. The 
Governing Board does not allow any type of massaging of the data, because people don’t 
view that as the proper approach. 



Mr. Kind. Thank you. 



Chairman Hoekstra. Thank you. Without objection, I would like to submit for the 
record a February 18 letter,- February 18, 1999, from Chairman Mark Musick to Mr. 
Forgione, just quoting, "I am writing to express the Board’s concern about the recent 
. press conference for release of the 1988 NAEP reading report card. We believe the 
format, tone and substance of that event was not consistent with the principle of an 
independent, nonpartisan release of the National Assessment data, an important and 
longstanding Board policy." That letter was written at the direction of the Executive 
Committee of the National Assessment Governing Board I and my colleagues here on the 
panel believe. 



SUBMITTED FOR THE RECORD, LETTER FROM MARK D. MUSICK, 
CHAIRMAN, NATIONAL ASSESSMENT GOVERNING BOARD, WASHINGTON, 
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DC TO DR. PASCAL D. FORGIONE, JR, COMISSIONER OF EDUCATION 
STATISTICS, WASHINGTON, DC, FEBRUARY 18, 1999 - SEE APPENDIX E 



Mr. Roemer. Mr. Chairman, I will not object to your request to enter that into the 
record, but I would make a request at this time to have unanimous consent to enter into 
the record right after that letter Mr. Boyd's letter about President Bush's problems in 
1992. 



Chairman Hoekstra. Absolutely. Without objection. 



SUBMITTED FOR THE RECORD, LETTER FROM RICHARD A. BOYD, 
CHAIRMAN TO MR. EMERSON J. ELLIOTT, ACTING COMMISSIONER, 
NATIONAL CENTER FOR EDUCATION STATISTICS, WASHINGTON, DC, 
JUNE 3, 1992 - SEE APPENDIX F 



Chairman Hoekstra. What we are trying to establish here is that there is a principle of 
independent nonpartisan release of National Assessment data, an important and 
longstanding Board policy. 

My colleagues on the other side of the aisle tend to belittle that point and say this 
is no big deal, but actually in reality it is a big deal and the Board has consistently driven 
home that point. I am concerned by the inferences that are being made by the other side 
of the aisle that this is something that isn't very important and we ought to just blow it off 
because this is a partisan event. You are refusing to characterize this. 

Mr. Roemer. Mr. Chairman, I think you are characterizing the witness' testimony today 
in an inaccurate fashion. I think Mr. Forgione has said that this was not a violation in 
fact, but it was a perception problem. That is what he says in his testimony. 

Chairman Hoekstra. The format, tone, and substance of that event was not consistent 
with the principles of an independent nonpartisan release of the National Assessment 
data. Mr. Tancredo. 



Mr. Tancredo. Thank you Mr. Chairman. Mr. Forgione, on March 23rd, the L.A. 
Times quoted you as saying that Gore's presentation of the results "can cloud the 
confidence people might have in the independence of the data." 
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Education Week quoted you as saying the following about the Administration. 
"We have had contentiousness over how to proceed, but I never felt anything that would 
be out of line. It has only been over the presentation of the research, not the design." 



Based on usual testimony and these quotes, it appears you feel strongly the Vice 
President should not have released the reading results.. Can you explain for us a little 
more about what you mean by contentiousness? 



Mr. Forgione. .Well, first I will say there is a subtle but important difference here. 

When I was asked the question, I do believe that- the way this was released in a press 
conference did have the perception of politicizing this event. Therefore, as I said in my 
openingstatement, we.have got to have a strong measure of independence, and therefore 
the Commissioner should have released the data at that event. It doesn't mean because we 
electronically released it, we do those things simultaneously. 



So I felt that this was not an appropriate approach. I believe it was f inadvertent, 
but it is something that we do not wish-to have continued. I am pleased the Governing 
Board has taken action with regard to NAEP that will not allow this in the future. I ask 
you in your reauthorization to ensure that this independence happens. 



At this moment, my Agency is suspect and people in the education community are 
raising major concerns. If you are goingrfo attract a new Commissioner to this position, 
you want someone who iscgoing to come in and not be afraid to do what is right. 
Therefore, the perception is at this moment that this thing didn’t go well. I have no 
evidence, but I just don't like those, perceptions. The only way to.deal with it is get the 
boundaries clear and.get away from these awkward situations, and that is what I meant to 
. say. I.didn't mean to offend anyone. I meant to say, what is honestly, as the Chairman 
has said, is the policy. 



' Mr. Tancredo. . Mr.Ghairman, the reason why I am concerned about this, and I do 
believe that it is appropriate for us to be looking into this today. During the time I spent 
at the U.S.. Department of Education, a little over 11-1/2 years as the Regional Director in 
Region 8^ there ..was one place that we could go and one organization that we could look 
to for solid and good answers to questions. There was a general feeling that when you 
saw NAEP results, there was a reliability factor there perhaps greater than any other. 
Everything else we dealt with, I TBS all the way down, was fraught, frankly, with 
problems that did make 7 ou question whether or not you could use the information 
reliably. 

NAEP, however, was different. I was always encouraging those states in my 
region who are not participants in NAEP to become participants because of what I 
believed to be, as I say, a very good program you can rely upon. So it is extremely 
disconcerting that anything could happen in this particular program that would cast a 
shadow of doubt, even the slightest shadow of doubt, on the credibility of the activity of 
NAEP and of the results. It really is about the only thing we have to rely on in this 
environment; this arcane and sometimes ethereal world of measurement and assessment. 
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So I yield back the balance of my time. 



Chairman Hoekstra. Mr. Roemer. 



Mr. Roemer. Thank you, Mr. Chairman. Mr. Chairman I just want to get on the record 
that I have some concerns what has been conveyed in the last couple of statements 
regarding what Mr. Forgione has said in his testimony. He has said the Commissioner 
must release all education data for the agency. This is a quote from his testimony on 
page 4. This can take the form of publication, a data file or an electronic release. The fact 
of the matter is it was electronically released an hour before the press conference. 



Then Mr. Forgione goes on to say on page 5, that the Commissioner electronically 
released the national NAEP data on the NCES web site at 10 a.m. that morning. 

However, there may be a subtle perception problem, which now Mr. Musick and Mr. 
Forgione have clarified internally. 



Mr. Musick, could you read for me your new rule that you published 2 weeks ago 
to make sure this doesn't happen again? 



Mr. Musick. I think I can, sir. I think reading the relevant parts of it are really what you 
want to hear. One second please. 



In revising the NAEP release policy earlier this month, the Board added new 
language which underlines our previous position. It reads: The initial release of NAEP 
data shall be independent and apolitical, separated clearly from other programs and 
policies of the Department of Education. 



It also adds that the official press statement or release announcing NAEP results 
shall be issued by the National Center for Education Statistics rather than by any other 
department in the Department of Education. 



Mr. Roemer. This is a result, Mr. Musick, of the perception problem that the Vice 
President inadvertently, and that is a quote from Mr. Forgione, inadvertently created 
even though the data was out on the web an hour ahead of time. Now a political person 
according to the guidelines can no longer be part of an announcement, and that is a result 
of this social faux pas by the Vice President and former President Bush's violation. Is that 
correct? 



Mr. Musick. This is the Board's effort to make sure that. . . 
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Mr. Roemer. ...to clarify... 

Mr. Musick. ...to clarify... 

, Mr. Roemer. Thank you, Mr. Musick, because you have addressed this. This seems to 
me to be a Committee :in search of a. perception problem. We just had the Chairman 
begin to criticize the Vice Presidents speaking style. . He said his presentation before Mr. 
Forgione was sterile or his. presentation. Is that our next Hearing? Are we going to start 
to have Hearings on the Vice President's speaking style and presentation? 

I mean, come on. We have better things to do with our time, with staff time, with 
this Committee. We have now, and this is not something I am making up, in Roll Call 
people saying that there are teams of Republicans that are going to read from the Vice 
President's book on the House floor, and that people are going to magnify every 
statement he makes. 



We wouldn't be here today if the Vice President was not in the running for 
. President. If he was a Vice President. simply performing his. Vice Presidential duties and 
he had done this, we probably would not be here. today. But because he is ^candidate for 
President on the Democratic side we are here. We are having a long Hearing, you have 
. addressed it internally, and what I worry about is that the GOP is going to turn. .. 

Chairman Hoekstra. Excuse me, Mr. Roemer. 



Mrr Roemer. It is my time, Mr. Chairman. 

... Chairman Hoekstra: Lbelieve that-you are characterizing my behavior and my 
motivation. 



Mr. Roemer. No, I am not. I just quoted you about what you said about the Vice 
• President's speech. 

■* Chairman Hoekstra. Yes, you are. You characterized this.. 

• Mr./Reemer. It is my time, Mr. Chairman. 

Chairman Hoekstra. You characterized this Hearing. . . 
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Mr. Ford. Mr. Chairman, the Ranking Member deserves his time. 
Mr. Roemer. It is my time. I would be happy to yield. 

Chairman Hoekstra. This is a matter of personal privilege. 



Mr. Roemer. I will be happy to yield to you when I finish my statement. 

Chairman Hoekstra. This is a note of personal privilege. I believe you are 
characterizing my motivation. 

Mr. Roemer. No, I am not. I would not characterize your motivation. 



Mr. Ford. I would. 



Mr. Roemer. I am quoting you and what you said in your statement about the Vice 
President's speaking style and I am quoting you in what has been in publications across 
this City, going after the Vice President. And we have now had a 2 hour Hearing. We 
have had Mr. Musick and Mr. Forgione tell us that they have cleared this thing up, and 
we have had Mr. Forgione say that this is an inadvertent perception problem. 

Chairman Hoekstra. The gentleman’s time has expired. 

Mr. Roemer. I would be happy to yield to the Chairman. 

Chairman Hoekstra. Let's keep going here. Dr. Forgione, you have not been 
reappointed to the Board is that correct? Excuse me, as Commissioner. 



Mr. Forgione. I made a decision to withdraw my nomination last Monday. 

Chairman Hoekstra. Can you give me a little background as to how you reached that 
decision? 



Mr. Forgione. I was informed by the White House that because I have made a mistake 
in the fact that I deliberately overpay my taxes each year. On April 1 5th, I let the IRS 
know that I am getting a refund, but I did not finish my taxes asking for the refund by 
August 1 5th. Each year I got a refund, I never got sanctioned by the IRS, never got 
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penalized. But they feel that this is a mistake that in their minds and it precludes me from 
being eligible for reappointment. 



The Secretary tried to intervene. He wasn’t successful. So I decided at this point 
my term is up on June 21st, that I should just proceed onward. 



Chairman Hoekstra. In characterization of this little social faux pas by the Vice 
President, did you have discussions or e-mails, or your staff back and forth with the 
Vice President as to exactly what was going on and how the process should work? 



Mr. Forgione. I had a conference call on Monday, February 8th, in which the 
Secretary's office, my office, and the Vice President's office talked about his intention to 
join us which we welcomed. At that point when I left the meeting I thought it was going 
to be at 10 o'clock as we had previously decided that I would present, Mr. Musick would 
follow, and then they would join us and have the kind of activities that would follow. 



When I left that meeting it was my understanding that was.going to happen. I was 
subsequently informed by one of my staff members the next morning that it would be 
different, and I had no idea that the Vice President would come and release the data at 
that point. I was not aware of what was going to happen. I had no conversations and no 
e-mails with the Vice President's office from that Monday to this day. 



Chairman Hoekstra. Did you clearly outline to them what the policy statements were? 
Did they have a.thorough understanding of what those policy, statements were as to how 
data should be released? 



Mr. Forgione. J always try to advocate for the importance of the, perception and fact of 
integrity and independence. The Department has always supported the Commissioner 
having that prerogative to release. If there are circumstances where it can't be done, then 
we can try to accommodate it. But when we have a press conference, I have always gone 
first, with the Governing Board and others later. This was as I said, just a departure from 
that procedure. 



Chairman Hoekstra. Were you led to believe that you would be reappointed as 
Commissioner? 



Mr. Forgione. Obviously, as I entered the last year of my term. It is a fixed 4-year term, 
and I was appointed in June of 1996. In January, the indication was that if I met the 
standards of the FBI check and the other types of scrutiny that I would be. 



Fourteen weeks into it on April 14th, I received this call. I was shocked by it, but 
I accept the fact that the President can appoint anyone that he deems appropriate, and I 
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take them at their word what the reason is. I am proud that the Department advocated, 
but it wasn't successful. 



Chairman Hoekstra. Mr. Musick, do you provide any feedback to Dr. Forgione as to his 
performance as Commissioner? 



Mr. Musick. Not formally, no. 



Chairman Hoekstra. What has Mr. Forgione's performance been? 



Mr. Musick. Working with the Commissioner through the NAEP program which of 
course is what I am most familiar with. Pat's open style and the way he worked with us 
on the redesign of the National Assessment has been to tell us what he is thinking about, 
to share information with us and to let us respond to drafts. It is a very open style and 
one for which I think the Board Members would give him high marks. 



Mr. Roemer. Mr. Chairman, you certainly don't have to yield to me, and I doubt if you 
will, but can I ask a question? Has this Hearing now gone from the NAEP test to whether 
or not the administration has the ability to reappoint political appointees? 



Chairman Hoekstra. The issue has moved from one of whether the administration 
politicized an event which is unacceptable, whether Bush does it or whether this Vice 
President does it, because the key to this being a successful organization is that they be 
independent. And I believe earlier today Mr. Forgione indicated that there was some 
question or some cloud over this organization based on what had happened in February. 



And I am now concerned that as Mr. Forgione has articulately and actively 
fought for maintaining the independence of that Agency and has done a good job, that 
perhaps by challenging and doing his job for maintaining the independence of that 
agency, may have lost the opportunity to be reappointed to this job. 



Mr. Roemer. Would the gentleman further yield? I think we have to be very careful 
now, Mr. Chairman, to go forward in a Hearing, public Hearing, to talk about the 
personnel practices of Mr. Forgione and start getting into tax returns and a host of 
different things. I don't think this is fair to him. I don't think it is fair to him to evaluate 
his performance which I am sure has probably, from what I can see, been good. But we 
are not equipped to start having a public Hearing on this gentleman's performance in 
front of the press and the public and everybody else. I don't think it is fair to him. 



Chairman Hoekstra. Thank you. Mr. Ford. 
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Mr. Ford. Sure. I think so much has been said, and pretty much everything has been 
said. The Chairman was questioning whether or not Mr. Roemer was questioning his 
motives, and I would say that you just questioned the Administration's motives and what 
action might have been taken with Mr. Forgione. 

I don't like questioning anyone's motives,,but when you leave yourself open for 
those kinds of questions you have to be prepared to answer them. Perhaps the 
Administration will have to answer this question. I think Mr. Chairman questioning the 
Vice President the way you have questioned him and the Vice President's office certainly 
can lead one to suggest or. one to reach the reasonable conclusion. I am fascinated by the 
legal terminology being used on the other side, "the intent." We are acting as if the man 
1 broke some sort of law by announcing what he read across the Internet. 

I would only.ask Mr. Forgione and Mr. Musick, if you wanted to make this 
announcement yourself? I understand why you would want to make the announcement. I 
. say I represent the City and I have two other colleagues that represent it with me, and 1 
can tell you when good news comes our way we are all rushing to announce the news. 

But if I release ifacross my web page and then hold a press conference to announce it, 1 
can't act like I am dumbfounded when my colleague announces it ahead of me. 

Mr. Musick if you were so opposed, 1 think I quoted you correctly. You sort of 
got sidetracked a bit when you said in your letter, the format, tone and substance of that 
event was not 1 consistent with the principles of an independent, nonpartisan release of 
National Assessment data. Those are your words, and 1 imagine you would link it to Mr. 

. Bush's comments as well as the question 1 was getting at. You pushed it off on Mr. Boyd 
but that is what you said. 

Why then did you participate in an event that you thought was so partisan or 
violated the principle or was not consistent with the principle of this independent, 
nonpartisan release of the National Assessment data? Because if my memory serves me, 
you said you participated along with this Fairfax teacher, the student and Vice President. 
Again, if this violated the principles of the Board why would you voluntarily participate? 

Mr. Musick. Yes, sir. As. you heard Commissioner Forgione say, the involvement or the 
change in the Board's plan took place over a 48 to 72 hour period. We were told there 
might be some change, that a teacher would be involved, the Vice President's office 
called and might be involved. That was on a Friday. 



On a Monday we were told it is looking more like it. When 1 arrived that morning, 
I did not know the order of the presentations. 



If you will also look at my letter Congressman, you will see in the next sentence 1 
said to Mr. Forgione, "You and 1 discussed this problem immediately after the release on 
February 10th." We were standing on the stage or just off the stage. So 1 didn't know 
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going into that morning how the event was to unfold. 



Mr. Ford. Would you have felt better had you been able to make the announcement and 
not the Vice President? 



Mr. Musick. I would have felt okay only if the Commissioner had made the 
announcement. 



Mr. Forgione. If I could comment, Mr. Ford. I guess an analogy I would like to ask you 
to think about is this: Would you be comfortable with the President releasing the 
unemployment statistics each month? Certainly the highest office in the Nation you want 
to give honor to. The question is, when you have important indicators that are so vital 
and so important to public policy you want to insulate that as I said in the statistical 
document. 



Mr. Ford. I don’t mean to interrupt. I know Mr. Roemer wants to say something, but let 
me ask you one last question. Do you think the Vice President manipulated the results? 
Do you think he somehow or other was able to penetrate your network or system and rig 
these results to show this? Because I am curious. I don’t quite get it. Maybe it is me. 
You put them out an hour beforehand. The results showed improvement and showed 
progress. Are we somehow suggesting that the integrity of this process, the 
independence of the Board, has somehow been threatened. You released it across the 
Internet, not the Vice President. The Vice President, or me, or whomever, I must tell you 
if you were to do it again before you announced it, I would make the announcement. I 
don't understand what the big deal is, Mr. Chairman. 



Mr. Forgione. We won't give you access to the data. 



Mr. Ford. Hold on one second, Mr. Forgione. I don't understand what the big deal is. I 
have got to think if Mr. Goodling or you had access to it, you might have made the 
announcement. 



To call these two fellows here, to depart from what we were talking about as Mr. 
Roemer said, to talk about this guy’s tax returns. Mr. Forgione seems like a great guy. I 
don’t know all the facts, but I would be happy to appeal on your behalf with the 
Administration if needs be, because you seem like an upstanding and honest sort of a 
“shoot-straight” kind of guy. 



But I don’t understand why we are so offended here. The Vice President didn’t 
manipulate the results. He didn’t go into your systems and figure out how to say, "Fourth 
and eighth graders are reading better. I have got to do this so my campaign or the 
Administration looks better." 
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All he did was announce results that you all put together and released an hour 
before his press conference. Mr. Musick goes to the press-conference, participates and 
then gets offended when he can’t go first or you are not there to release the. results before 
the Vice President inadvertently does,after you had already done it on the Internet. 

• When someone questions your motivations Mr. Hoekstra, it is because of this. 
Believe me, if there was something the Vice President had done wrong, 1 would be the 
first to say. But I don’t see what he did wrong here, other than go on the Internet, find the 
results, call a press conference, invite the Board, and make the announcement. 

Perhaps it is me, but I can tell you if you do it again and 1 see it before you guys 
do r it,** you 7 may-have to hold a Hearing on me because 1 will make the announcement as 
quickly as the Vice President did. With that; I yield to Mr. Roemer whom I know wanted 
to ask- a question as well. 

- Chairman Hoekstra. Mr. Castle. Are you ready? 



Mr. Castle. No, sir. 



Chairman Hoekstra. Mr. Schaffer. 



Mr.. Schaffer. Thank you, Mr.Chairman. Any charge made that the Vice President 
manipulated the data is not unfounded, given the comments that you made a little bit 
earlier. It is all in the presentation of the data and the statistics. 

-There is the phrase. I can't remember how it goes, but basically there are lies, 
statistics, or something? 

Mr. Forgione. Lies, big lies, and then there is statistics. 

- Mr. Schaffer. There are lies, big lies, and then there is statistics. That may apply here as 
well. Because- what the Vice President referred to was really a subset of the overall 
statistical data, with total ignorance of the statistical baseline to suggest whether student 
scores actually indicate any kind of improvement at all. Again, the baseline used by 
NAEP is 1992 to 1998, am I correct in that? 



Mr. Forgione. Yes. 
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Mr. Schaffer. The Vice President used the time frame from 1994 to 1998. So is it fair 
for me to say that the President told the truth? 

Mr. Forgione. They certainly presented half of the truth. 

Mr. Schaffer. But he didn't tell the truth, right? 

Mr. Forgione. Not the whole truth. 

Mr. Schaffer. This is my next comment. It is fair to say the President didn’t tell the 
whole truth. 



Mr. Roemer. Are we talking about President Bush or Vice President Gore? You keep 
referring to him as the President. 



Mr. Schaffer. Excuse me. When I talk about not telling the whole truth, it is hard to 
keep track of which one I am talking about in that case. The Vice President is what I am 
referring to in this case. It is accurate to say the Vice President did not tell the whole 
truth with respect to disclosing NAEP’s data. 



Mr. Forgione. I prefer to say the Vice President presented part of the data and the 
Commissioner later presented all parts of the data. I would say that is not trivial and that 
is why we want the Commissioner to go first, because America has to believe the data. 
What he did was not inappropriate, if in fact all of the information had gotten out. When 
the Vice President leaves, a lot of the press leaves with him. 



Mr. Schaffer. I would agree with you that this is not trivial. 



Let me ask, Mr. Musick, why do you suppose you were invited to be at this 
event? This announcement where this information was disclosed and the Vice President 
shows up, and you said the teacher was there. What do you suppose your purpose was 
there and why you were invited? 



Mr. Musick. I was there as Chairman of the Board. The event was as you have heard, 
and the plan was approved weeks in advance. That is how we do it. 



Mr. Schaffer. Let me stop you there. Were any other Members of the Board invited to 
participate in the role you were invited to participate in? 
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Mr. Musick. Normally we only have one Member from the Board. We rotate and try to 
give different Board members a chance. 



Mr. Schaffer. Having a Board Member there in your opinion would serve what useful 
purpose? 



Mr. Musick. The Commissioner presents the facts and normally the Board Member 
most closely affiliated with the facts makes the first comment. 



Mr. Schaffer. Do you believe the credibility of.the Board is predicated on the 
.nonpartisan nature of board members and the Charter of the Board? 



Mr. Musick. You think the Board is nonpartisan. We have elected officials on the 
Board, but I would argue the Board operates in a nonpartisan way. 



Mr. Schaffer. So the nonpartisan credibility that the Board . maintains, do you believe 
that level of credibility has some useful purpose in presenting the data, the numbers and 
so on? 



Mr. Musick. The nonpartisan Board sets the policy for-NAEP.and the Commissioner 
implements it. So they^both have4o be nonpolitical; nonpartisan. 



Mr. Schaffer. The Vice President, prior to announcing the half truth, went through and 
said a number of things. I am quoting.' "We have worked hard, very hard, to help states 
and communities raise academic standards, invest in teacher training,: and marshal the 
effects of thousands : of young people through the AmeriCorp program and also an 
expanded^work- study program. And for all of these reasons; under the leadership of 
President Clinton, education in America is America’s top priority. Well, today l am very 
..proud to be able to report to you the new.evidence that our efforts are beginning to pay 
off," and so on. 



So there is.the Vice President making the claim thathis bureaucracy and his 
programs and.so on have. resulted in an increase in scores about which we should all be 
excited, andthey invite you to.stand'next to him as a credible member of a nonpartisan 
Board. 

* Let me just ask: Isthis why you are uncomfortable? Do you think that this whole 
scenario suggests an effort to politicize the data? 



Mr. Musick. This whole. scenario? I think this event was an unfortunate deviation from 
our Board policy. I do not believe that the National Assessment's credibility today is 
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terribly impugned. I think this was one event. We had one other event we talked about. 



Mr. Schaffer. Do you think the Vice President took advantage of your credibility in 
attaching your statistics to his programs? 



Mr. Mustek. I would not, and I could not answer that question. 



Chairman Hoekstra. The gentleman's time has expired. We are going to yield to Mr. 
Roemer for a question. By unanimous consent, I believe unanimous consent, we are 
going to allow Mr. Castle to ask some questions and then we will end the Hearing. We 
have another vote coming up. 



Mr. Roemer. Mr. Musick, just so I understand where we have gone in the last 2 hours 
and 10 minutes. If the teacher from Fairfax, what is her name? 



Mr. Musick. Emily Parker, I believe. 

Mr. Roemer. Emily Parker. If Emily Parker had introduced Mr. Forgione first to 
announce the statistics and then Vice President Gore had followed Mr. Forgione, would 
we have had a perfectly okay perception of this event then? 



Mr. Musick. I think that if that had been the plan put before the Board and approved, 
and had been followed in that way, we probably would have. The comments that were 
made were interpretive comments by the Vice President, and we have held to present the 
data, and not comment on it. So probably so. 



Mr. Roemer. Thank you. 



Chairman Hoekstra. That is a recess. We have all the time in the world now. Mr. 
Castle. 



Mr. Castle. Thank you Mr. Chairman, for yielding me all the time in the world. I can 
use it for all these questions. 



A couple of statements; I believe this Hearing is worthwhile for an entirely 
different reason. I believe that the process of testing is overwhelmingly important and 
the equity and fairness of testing is of overwhelming importance. 
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We are going to be reauthorizing NAEP and NAGB. That is never easy. We have 
been through it before, and for that reason I think it is important that we straighten .even 
some of these things out. 



I think you all know my feelings about this. I think when you give these tests 
and you say they are statewide results, or you break it down by state, then doggone. it, it 
ought to be fair in terms of the balance of those with learning disabilities, as well as those 
who are good students in private schools, or whatever it may be. If that is the claim; let's 
make sure it is happening. I know it is complex. 



That much I learned when I was on the Board. I never did understand it all, but I 
know it is complex. I just beg and beseech you all to do it. I know some of your good 
staff are behind you in that, too. 



Secondly, I hope Dr. Forgione in the case of you not being reappointed, that it 
was for valid reasons. Frankly, I have heard this tax business. That doesn't seem 
particularly overwhelming to me in light of some other appointments and reappointments 
in this and other Administrations in fairness. I will just let it go at that. I just hope it was 
not for political reasons. I am not stating it was. I hope it didn't work out that way. 



I am going to ask you a question, Mr. Musick. I didn't start on this whole political 
kick, but I have become concerned. I understand there is a memo or something that 
shows that the Vice President was not to be the person to lead this off. 



Maybe you answered these questions when I was doing an announcement on 
teachers at a press conference, but at what point was a decision made and who made the 
decision to allow the Vice President to be first? I don't think he should have been 
anywhere near it. I don't think President Bush should have done what he did years ago. I 
don't think any politician should be anywhere near an announcement like this. 



My question is, how did this happen? Does this impact the integrity of NAGB in 
some way or another, in terms of how you all operated that particular day? I understood 
that afternoon before he was not to be first. Somehow the order got changed. 



Mr. Musick. I think Commissioner Forgione did not have any direct comment. I have 
said to Congressman Ford that I walked in the room that morning and found things not 
exactly as I had anticipated. 



Mr. Castle. You weren't involved in that? 



Mr. Musick. There were conversations over a 48-hour period. We were told the Vice 
President would be involved, likely he would be involved, or the Vice President is going 
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to be involved. And the Board, as Mr. Roemer raised a moment ago regarding the Vice 
President’s involvement at that time in an appropriate way, viewed it as bringing more 
attention to this. 



I think the question you raised a moment ago Mr. Castle, is a good one. Maybe 
we need to ask the question: Can elected officials at that level be involved without it 
skewing the process? 



Mr. Castle. The answer is no. 



Mr. Music k. The Board has not taken that up. 

Mr. Castle. Why did Emily Parker introduce the Vice President? Who told her to have 
it happen that way? How did that person come to that decision? Does anyone know? 
Did anyone ask her? Do any of you know how? Have you asked the question how it 
came that he did go first? Obviously that is in real contradiction to your own rules and 
regulations that that would happen. I would hope that somebody has raised the question. 
I would like to know the answer of how this came to happen that the Vice President was 
the one that started this off. 



Mr. Musick. There were also questions about time. Commissioner Forgione mentioned 
originally it was going to be 10 o'clock, then 10:30 or 1 1 and then back to 10. I assumed 
when I was hearing that, that different persons' schedules- were affecting this. I assumed 
that this was a Departmental decision, but the Commissioner, again, I had no... 



Mr. Castle. Dr. Forgione, you don't know either? 



Mr. Forgione. After I left the meeting on Monday, when I thought we were going to go 
at 10 and they were going to come in at 11,1 have no knowledge of who made the 
decision and why. We were just informed of it. 



Mr. Castle. It could have been made at a Departmental level, at a higher level within the 
Department. Wouldn't they have to communicate it to one of you? 



Mr. Forgione. They communicated the fact and we prepared the agenda. 



Mr. Castle. Somebody communicated to you the fact that the Vice President would go 



first? 
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Mr. Forgione. My press person got called and told this will be the order of the sequence. 



Mr. Castle. It was the Department of Education that made the decision that the Vice 
President would go first, or at least they communicated that to you? 



Mr. Forgione. Yes. 



Mr. Castle. We can only assume how it got to them. Okay. I have no further questions. 



I will just reiterate what I said. I think it has been a problem. I think we really 
need to address it. Hopefully this incident will eliminate the politics of it and you can 
concentrate on the job you are doing. You do your job well, but this is a little bit of a 
mark against it. 



I yield back, Mr. Chairman. 



Mr. Forgione. Mr. Chairman, Mr. Schaffer, I believe I misspoke a few minutes ago, 
characterizing what the Vice President said as a half truth. I think it is an interpretation 
of the data which would be legitimate in a context. It did happen. But the context has to 
be correct for it to be there. So I think I misspoke on that. 



Mr. Schaffer. Was the context correct? 



Mr. Forgione. I think there was a fuller context. 

Mr. Schaffer. But was the context correct in the case of the Vice President’s 
presentation? 

Mr. Forgione. The Governing Board has determined in its letter that this practice should 
not happen. 



Chairman Hoekstra. Mr. Ford, I believe, has a question. 



Mr. Ford. Let me just say two things. One, I have enormous respect for Governor 
Castle. I am a little surprised that he would sort of take the line of questioning that he 
did. But in fairness to the Governor and the fact I know the Scheduler and Director of 
Protocol at the White House for the Vice President, perhaps we can subpoena them or 
have them come before the Committee to have them answer all of these questions fully 
and get to the bottom of what is emerging as a huge scandal as to how decisions are made 
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at the White House and in the Vice President's office with regard to who speaks first and 
who speaks last. 



I will say that the next time anyone from NAEP goes over, and 1 have a few 
friends over in the Office and perhaps we could have saved time and resources here 
today, I will assure you that never again will the NAEP be slighted at the White House or 
any event the Vice President is participating in. I feel confident saying any event you 
participate in with this Committee never again will you be slighted. I hate to sound petty, 
but it sounds as if that is where we are going and where we have been going for some 
hours now, and I am contributing to this by prolonging this. 



My question was not answered because I did so much talking last time. Do you 
think the Vice President manipulated the data, either one of you however he interpreted 
it? Forget the interpretations for a moment. We have some data here that the Board 
wanted to release and, in fairness to the Board, should have spoken first. But since you 
didn't, do you think that the source of the misinterpretation was the data that the Vice 
President interpreted? Do you believe that he had anything to do with producing or 
generating those results or those numbers, or the actual data that the Board was able to 
collect? Mr. Musick? 



Mr. Musick. You have a two-part question. 



Mr. Ford. No, it is one part. Do you think he manipulated the data? 



Mr. Musick. One, I do not. This is the reading report on the desk here. There is no way 
anyone could have presented all the data. 



Mr. Ford. Do you think he had anything to do with manipulating, putting together the 
data, or collecting the data? 



Mr. Musick. No. 



Mr. Ford. Mr. Forgione, do you think the Vice President had anything to do with the big 
book you just raised there? 



- Mr. Forgione. Absolutely not. 



Mr. Ford. Okay. Now, we talk about politicizing. The politicizing we are talking about 
he put, for lack of a better word, his spin on the report that you guys released. Now I can 
assure you, you are going to get 435 different spins here. You are going to get the 
number of Members on this Committee to spin, which we do all the time. Why are we 
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acting surprised that the Vice President might have a different perspective on these 
numbers than Mr. Hoekstra or myself, or Mr. Roemer or Mr. Castle? It is a little bit 
surprising. 



Are you surprised that maybe two different people, particularly folks who have 
different political philosophies, might interpret that data differently? 



Mr. Forgione. No, I am not surprised. 



Mr. Ford. Are you surprised, Mr. Musick? 



Mr. Musick. No. 

Mr. Ford. Do you guys find yourselves interpreting things differently in your office, the 
times you are able to interact on issues? 

Mr. Forgione. We don't interpret. 

Mr. Ford. Do you disagree ever, you and Mr. Musick? 



Mr. Forgione. Seriously, we give you the trusted data and allow you to make it. The 
Governing Board can interpret. It is not a function of a statistical agency to make those 
judgments. 



Mr. Ford. So in light of what you have just said, if you were able to have spoken first at 
this event which we are so concerned about, and I do hope that someone is trying to find 
out the name of the Protocol Director and the Scheduler to make sure we don't have this 
problem again with the White House, what is it that you would have said differently? 
Since you can't interpret the data, how would you have presented the data any differently 
than the Vice President? 



Mr. Forgione. I would have presented you the full set of data. Again, I give you the 
analogue: Would you be comfortable with a political official releasing the unemployment 
rate or the Consumer Price Index? That release is important. It goes to integrity and 
independence. It may just be a procedural activity, but it is much more than that. You 
created a statistical agency because you want... 



Mr. Ford. Why did you put it out on the Internet? 
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Mr. Forgione. The Vice President didn't get this on the Internet. 



Mr. Ford. He could have conceivably gotten it across the Internet, because you put it 
out. You released it electronically. 



Mr. Forgione.. This was the event the press was coming to, and education groups trying 
to see have our children made progress? A very important social indicator. That has to 
go out completely and carefully. 



Mr. Ford. Did you put it out electronically completely? 



Mr. Forgione. Absolutely. The whole document. But it is not easy to get the message 
out of this big document that you can get from a clear statement by someone. Don't 
underestimate your power and the power of officials. That is the organization. I believe 
the Governing Board has solved this for NAEP and F hope. you will solve it for the rest of 
NCES. 



Mr. Ford. Thank you. I would say to my friend Mr. Hoekstra, thank you for 2-1/2 
hours. I hope we get the same amount of time to deal with Conyers, Georgia and hope 
we get the same amount of time with a reasonable discussion with Juvenile Justice here 
in this Congress very soon. 



Mr. Castle. Would the gentleman yield, in fairness, Harold? 



Mr. Ford. Absolutely, Mr. Castle. 



Mr. Castle. Because now you.are getting into areas that concern me, because we have 
had substantial Hearings and Markups of Juvenile Justice and on the issues of violence. I 
don't want.to give anyone.the impression that this is the only thing we are focusing on. 



Mr. Ford. Mr. Castle, you know as well as I do sir, that we have not been able to bring it 
to the Floor because your leadership has not agreed to bring it to the Floor. 



Mr. Castle. I do know that. I am just suggesting probably too much time has been 
applied to it. I would love to get it to the Floor. So would you, probably. 



Mr. Ford. We. have had one Hearing sir, to my recollection, where we invited the young 
people to come in and give us their thoughts and ideas on this issue. I would hope that 
that would not be only one, because I think you and I agree on the severity and gravity of 
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this problem. 



Mr. Castle. There were three Hearings on that by the way, not one Hearing. 



Mr. Ford. We have lost more children in school hallways and cafeterias and school 
yards in the last 60 days than we have in the entire Balkan conflict, Mr. Castle, Governor. 

I would hope that three Hearings, as many Hearings as Mr. Burton had to get to the 
bottom of the campaign finance mess, as many Hearings'as we had to get to the bottom of 
the Teamsters mess. I think we owe it to our children to have a few more Hearings on 
this issue, and perhaps to urge the Leadership on both sides to rush this matter to the 
Floor, to allow discussion on these and other germane issues as it relates to children. 



The Vice President has been discussed fully today. Hopefully our children will 
have an opportunity to be heard in the coming weeks. With that, 1 yield back. 



Chairman Hoekstra. You yield back your time, great. Do you have a closing comment 
Mr. Roemer? 



Mr. Roemer. Do I have a closing comment? Yes. 1 would just say, Mr. Chairman, that 
first of all Mr. Castle and I have worked on a host of things together in a bipartisan way 
passing the education flexibility bill. You and 1 have worked together in a host of 
different ways. We have done a Hearing in Chicago on Public School Reform and the 
successes in public school reform. We are cooperating together on the Prison Industry 
Reform, and I would hope that those efforts would not only continue but be a priority in 
this Congress. 1 hope that we would continue to have field Hearings in places like 
Paducah and Pearl and Jonesboro and see what these communities have done in light of 
the terrible violence that has taken place in those communities so that we can visit with 
sensitivity, with bold ideas, with ideas for parents to get more involved, with ideas for 
school discipline, and with ideas for media violence. 



And, yes, maybe guns are a part of it. Maybe gun safety for children is part of the 
solution. Guns wouldn't be in my top 3 of 4, but it would be something that 1 hope we 
address soon in the United States Congress. 



I don't think that we can have a more important thing to address in this Nation 
with the oversight of this important Committee that has the ability to go in and see under 
your Leadership a host of different things on oversight and investigations: school safety, 
school violence, how we use the bully pulpit to try to keep our families together. We 
can't legislate that. How we try to maybe embarrass our media companies into doing less 
of the video games that are so destructive for some of the young people. We probably 
can't legislate that with the first amendment. 
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If we could spend a day on the House Floor on an emergency supplemental for 
our troops overseas, I think we can spend weeks and months on what is the most 
important thing in the country right now to the business community, to the unions, to the 
people, to the farmers, that we need to address public education reform and school safety. 



We now have the confluence of two issues coming together in this country: Bold 
public school reform and school safety. Gallop polls across the country are seeing 10 and 
15 and 20 percent increases in the amount of concern expressed by parents that they don’t 
feel that their kids are safe in American schools anymore. 



It seems to me that that is the most important issue we can face in this country, 
along with public school reform. I would hope that under your Leadership and Mr. 
Castle’s leadership, people that we have worked with before on these issues, that we will 
get to those issues as quickly and thoroughly and for as long as it takes to address in 
bipartisan and substantive and thorough ways, some answers to these very deep and 
difficult questions that the American people are asking us. There is nothing more 
important right now. 



I would thank you for your indulgence with that statement. 



Chairman Hoekstra. Thank you. 

Mr. Forgione, Mr. Musick, thank you very much for being here. I can tell you 
that Mr. Castle and I and this Subcommittee, at least on its Majority side, applaud your 
efforts and your vigilance recognizing that your credibility and the credibility of the data 
that you provide is only as good as the perception that you are working independently in 
providing us data that reflects that independence. I am sorry for some of the inferences 
today that maybe that independence is not necessarily all that important, or that this data 
can be used and can be politicized. As we go through and Mr. Castle goes through the 
reauthorization of these programs next year, you can be sure that we will probably 
institutionalize and legislate a greater degree of independence. We will take a look at 
some of the other data-gathering agencies of the Federal Government and see how they 
are protected. 



So rather than moving in a direction that potentially politicizes these events in the 
future, we give you a greater degree of independence to do the job as you best see fit. 
That is the direction. That is the commitment that I will make to you and I believe we 
will see in the legislation that Mr. Castle will shepherd through his Subcommittee and 
through the Committee and through the House. 

Thank you very much for being here today. There may be some questions yet 
which we haven’t had time to ask. I would like to submit those questions in writing so 
that we can insert your responses as part of the record. Without objection, so ordered. 
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SUBMITTED FOR THE RECORD, LETTERS FROM CHARI MAN PETE 
HOEKSTRA, SUBCOMMITTEE ON OVERSIGHT AND INVESTIGATIONS, 
COMMITTEE ON EDUCATION AND THE WORKFORCE TO DR. PASCAL D. 
FORGIONE, JR, COMMISSIONER, NATIONAL CENTER FOR EDUCATION 
STATISTICS, WASHINGTON, DC AND MR. MARK D. MUSICK, CHAIRMAN, 
NATIONAL ASSESSMENT GOVERNING BOARD, WASHINGTON, DC, JUNE 2, 
1999 WITH RESPONSES FROM EACH DATED RESPECTIVELY JUNE 14, 1999 
AND JUNE 9, 1999 - SEE APPENDIX G 



Chairman Hoekstra. The Subcommittee will be adjourned. 



Whereupon, at 12 noon, the Subcommittee was adjourned. 
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APPENDIX A - OPENING STATEMENT OF CHAIRMAN PETE HOEKSTRA, 
SUBCOMMITTEE ON OVERSIGHT AND INVESTIGATIONS, COMMITTEE ON 
EDUCA TION AND THE WORKFORCE 
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SUBCOMMITTEE ON OVERSIGHT AND INVESTIGATIONS 
COMMITTEE ON EDUCATION AND THE WORKFORCE 
US HOUSE OF REPRESENTATIVES 

2181 RAYBURN HOUSE OFFICE BUILDING 

WASHINGTON, DC 20515-6100 
Opening Statement 

CONGRESSMAN PETER HOEKSTRA 
Chairman, Subcommittee on Oversight and Investigations 

Review and Oversight of the 1998 Reading Results of the National Assessment of 
Educational Progress (NAEP) - the Nation’s Report Card 

May 27, 1999 

We are here today to address the politicization of the National Assessment of 
Educational Progress and the trustworthiness of the scores that states received during the last 
reading assessment. As the Nation’s Report Card, the National Assessment of Educational 
Progress, or NAEP, is an ongoing, Congressionally-authorized program that collects data 
through surveys on the educational performance of students in America’s elementary and 
secondary schools . The National Assessment is carried out by the Commissioner of Education 
Statistics under the policy guidance of the National Assessment Governing Board, or NAGB- 
The Board was created in 1988 upon the recommendation of the Alexander-James 
Commission, which urged that policy making for the National Assessment be buffered from 
manipulation by any individnal, level of government, or special interest within the field of 
education. 

In 1990, NAGB adopted its “Policy Statement on Reporting and Dissemination of 
National Assessment Results." This policy statement was - and in my opinion still is - 
straightforward and unambiguous. Specifically it states that the “reporting and dissemination 
of National Assessment results shall be insulated from partisan political considerations and 
processes." In addition, the “public reporting and release procedures for NAEP shall be 
apolitical, similar to those of other Congressionally-raandated data-gathering programs." This 
policy is necessary to ensure that the results, and indeed the entire testing process, is based on 
independent judgements that are free from inappropriate influences and special interests. 

On November 6, 1998, the National Center for Education Statistics submitted its 
“release plan" for the 1998 NAEP reading test. The release plan stated that the press 
conference would tak e place in early February 1999 and would begin with introductions and 
data presentation by Commissioner Forgione, followed by comments from Secretary Riley and 
a member of NAGB . This release plan was consistent with Board policy that stipulates that 
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“reports of NAEP results shall be issued by the Commissioner of Education Statistics. ” It was 
approved unanimously by NAGB on November 21, 1998. 

On February 10, 1999, Vice President Gore was the featured speaker for the release of 
the 1998 NAEP reading results, the first time any official higher than the Secretary of 
Education has participated in a NAEP release. The Vice President commented on several 
Administration initiatives during the release including: increased funding for Even Start and 
Head Start, connecting every classroom to the Internet, funds for schools construction bonds, 
100,000 new teachers, and AmeriCorps. The point is not whether I am for or against the 
programs the Vice President was talking about - it’s whether or not his speech was “insulated 
from partisan political considerations” as the NAGB policy stipulates it must be. In addition, it 
seems as if the Vice President wasn’t telling the whole story about the test scores. He noted 
that for the “very first time, reading scores have improved for each of the three grades 
measured by the National Assessment of Educational Progress” and that “this is great progress 
and we’re proud to report it. ” What the Vice President did not mention was the fact that 4* 
grade Achievement Levels showed “no statistically significant change,” 4* grade Scale Scores 
were “constant,” and analysis from 1992 to 1998 in the “12* grade shows no net gain over the 
period.” 



During the press conference, the Vice President made quotes about “a down payment 
on hiring 100,000 new teachers to reduce class size...” He noted that the Administration has 
“worked very hard to help states and communities... marshal the efforts of thousands of young 
people through the AmeriCorps program...” He even encouraged Congress to “enact the 
President’s agenda to improve education...” The Vice President’s participation was not an 
apolitical release of education data, but rather it seems as if this was an orchestrated media 
event to promote a political agenda. E-mails from the Office of the Vice President described 
the event as a “VP education event” that was “good for the VP, for the Department, and for 
education.” The Secretary’s office made suggestions about what programs to mention, such as 
the Reading Excellence Act - stating u It’s a reading day , why not push for reading dollars?” 

According to Mr. Musick - a witness before the Subcommittee today - “the format, 
tone, and substance of that event was not consistent with the principle of an independent, non- 
partisan release of National Assessment data.” Indeed, Education Week reported last March 
that “Vice President A1 Gore jeopardized the integrity of the tests when he announced [the] 
1998 reading results to a campaign-style rally...” The Vice President’s appearance - as the 
featured speaker before the Commissioner - differed considerably from the NCES release plan 
the was adopted by NAGB. 

Chairman Goodling expressed his disappointment because of the politicization of the 
announcement in a letter sent to Secretary Riley on March 17, 1999; In his response, the. 
Secretary claimed that Chairman Goodling ’s “concerns about the release of this data are 
unwarranted. ” With all due respect to the Secretary, I disagree with his assessment. I share 
Chairman Goodling ’s concerns and felt that the events surrounding the release warranted 
further review by the Subcommittee on Oversight and Investigations. 
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The Committee has received materials from NAGB and NCES that reveal a blatant 
disregard for the Policy Statement on “Reporting and Dissemination of National Assessment 
Results.” According to information submitted to the Committee, Commissioner Forgione 
participated in a conference call on February 8, 1999, with representatives from the Office of 
the Vice President, the Office of the Secretary of Education, and the Department’s Office of 
Public Affairs. It is our understanding that at that meeting, .Commissioner Forgione supported 
an agenda in which he would open the press conference and release the results at 10:00 A.M., 
followed by the Chairman of NAGB. Finally, the Vice President, the Secretary of Education, 
and a teacher would be given an opportunity to comment on the results. Subsequent to that 
meeting on February 8 and before the public release of the results on February 10, the 
Department “informed” NCES that the agenda had been changed. There was to be a shift in 
the order of the presenters and a delay until 11:00 A. M., at which time the Secretary and Vice 
President would open the press conference. At a minimum, the approved NCES release plan 
was disregarded. Subsequent to the press conference on February 10, NAGB has adopted new 
language for their policy on the dissemination of results in an effort to clarify the necessity for 
the credible release of National Assessment data. As I mentioned before, I believe that the 
original policy adopted by NAGB in 1990 was explicit and unequivocal. 

When it comes to making important decisions about the NAEP test itself, NAGB should 
be completely isolated from politics. When it comes to measuring the progress of our nation’s 
children, we need to ensure that scores are as accurate as possible and we must not let politics 
influence our ability to measure student achievement. In addition to questions about the release 
of the scores, I have questions about how well the new NAEP inclusion policies are working. 
Why did the exclusion of students with disabilities and limited English proficient students 
increase since the 1994 test? Was NAGB acting independently and in the interest of getting 
valid test data when it decided to adopt a new inclusion policy for the 1998 reading test? Was 
NAGB pressured into making decisions that have brought us to where we are today, 
questioning the way in which the results were released and the possibility that the scores 
themselves might have been clouded by politics? These are questions that must be answered to 
understand what we need to do to protect the NAEP test from political manipulation. 

Specifically, we should review the process in which the Vice President became the 
featured speaker; discuss whether or not the Vice President’s comments were thorough and 
consistent with Board policy; and revisit the issue of whether or not it’s possible for NCES and 
NAGB to remain independent as currently configured. 

I would like to thank our witnesses for appearing before the Subcommittee this 
morning, and with that I would yield to the Ranking Member for any statement he may have. 
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Issues Surrounding the Release of the 1998 NAEP 
Reading Report Card 

Pascal D. Forgiono, Jr. 

Good Morning. My testimony will focus on the two issues identified in your recent 
correspondences, namely: (1) whether some states possibly unproved their 1998 NAB? 
reading scores beca u s e they excluded higher proportions of students with disabilities and 
limited English skills than in previous years, and (2) were NAEP policies and processes 
followed (or not followed) in releasing the 1998 NAEP reading scores. 

Specifically, my testimony responds to your letters of April 21, 1999, April 29, 1999. and 
May 19, 1999 (see Attachments A, B, and C) which requested information ftom the 
National Center for Education Statistics (NCBS) and the National Assessment Governing 
Board (NAGB) related to the release of the 1998 reading results from the National 
Assessment of Educational Progress (NAEP), The 1998 NAEP reading results wen: 
released in two stages earlier this winter. On February 10* the National results for 
GradeM. 8. and 12 were released, and on March 4* the State results for die 43 
states/jurisdictions that participated in the grade 4 reading assessment and the 41 
statcs/jurisdictions that participated in the grade 8 reading assessment worn released. 

To respond to each of the eleven (l 1 ) items specified in your April 2l rt correspondence, 
two volumes have been prepared and provided to the Committee, one each by NCBS and 
NAGB. The NAGB responses (whieh arc contained in Volume 2) were transmitted on 
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May S* to your office and NCE5 responses (which axe contained in Volume 1 ) were 
provided on May 13*. 

My testimony is organized in two sections that provide background for the eleven items 
specified in the April 21* correspondence. First, 1 will set a context for the issueofihe 
NABP reading results and the rates of exclusion, and second, I describe the plana and 
timetable for the release of the 1998 NAEP r**etnig results. 
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I. State NAEP and Exclmlon Rates 



Issue Background 

Shortly after the release of the State NAEP results in early March, I was informed that 
concerns had been expressed around a possible relationship between changes from 1994 
to 1998 in the NAEP reading scores for some states and changes in the rates of exclusion 
of students with disabilities or students with limited English proficiency from 1994 to 
1998. The basic question is whether the changes in reading scores truly reflect overall 
differences in student performance, or whether they reflect to some degree, changes in 
the population being tested. The issue was reported in a number of national and local 
media. 

My office also received a number of inquiries from interested states that wished more 
information on this topio (e.g., Alabama, Connecticut, Kentucky, and West Virginia). 
Simultaneously, I received inquiries from a number of U.S. Department of Education 
staff who had become aware of this matter. 

The week following the release of the State NAEP reading results on March 4 fll , my staff 
in the Assessment Division began to receive a series of telephone requests for 
information on t his topic. I immediately met with staff to research this matter and to 
prepare for the requested telephone interviews with various media sources. I developed 
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on outline of the issue* to bo addressed and secured copies of the respective, tables from 
both the 1998 and the 1994 NABP Reading Report Card publications. 

I immediately contacted Dr. Roy Tniby, the Executive Director of NAGB, to advise him, 
and asked him to elect his staff and the NAGB of this emerging issue. 1 also informed 
hhn that I had directed Dr. Peggy Cast, who is the Associate Commissioner for the 
Assessment DivislotVNCBS and in charge of the NABP P r o gram , to proceed immediately 
to design a series of post-hoc analyses to explore the issue of the impact of the rates of 
exclusion on the 1998 State NAEP reading results. This would be an examination of all 
states that participated in the Grade 4 State NABP reading assessments in both 1994 and 
1998. 

I proposed Out the Commissioner brief NAGB on this topic at its upcoming Boon! 
mooting, scheduled for May 14, 1999. I. further indicated that it would never be possi bio 
to know precisely the impact of increased exclusion rates on NABP scores since such an 
Analysis Includes estimating what might have occurred had the exclusion rates been 
comparable. In addition, variations in state and local inclusion policies; as well as many 
other factors unrelated to inclusion, affect academic performance. Nevertheless, we are 
hopeful that these analyses wtfl shed some light on this issue. 

,Dr. Carr and her staff mitjatcd planninc work with our. primary NAEP contractor, the 
Educational Testing Service (ETS). Our initial analysis showed that this, issue was 
limited -to tho individual state samp lcs drawnffbrlhc 1998 NAEP reading assessment in 
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Crado 4. Since Ac 1998 State NAEP reading assessment represented the first time that 
State NAEP was offered at Grade 8 no trend data are available to bo analyzed, fn 
addition, since Iho NAEP national sample, which was drawn separately from the state 
samples, exhibited a comparable exclusion rate of 6 % for students with disabilities in 
both assessment years (1994 and 1998), changes in the exclusion rate can be plausibly 
rulod out as an explanation for the reported achievement gains for the Nation as a whole. 



To research the concern about exclusion rates influencing changes in NAEP reading 
scores in ono or more stales. NCBS asked ETS to conduct a series of analyses. Tho 
analyses examined reading scores and exclusion rates in all jurisdictions that participated 
in the NAEP Grade 4 reading assessment in 1998 and 1994, in order to address the 
following questions: 

1. What Is the pattern of changes in exclusion rates between 1994 and 1998? Is 
there an emerging pattern across all states or is the condition limited to a few states? 

2. What is the statistical relationship between changes in exclusion rates and gains 
in reading scores? 

3. Are there differences in the severity of disabilities of excluded students (n 1994 
compared with 1998? 

4. Were there differences in the exclusion rates in 1 998 between the disabled 
students who were offered test accommodations aod students who were not offered 
accommodations? 



5. For states with the largest increases in exclusion rates, how would gains in 

71® 8C0r0 * have been affected. If 1994 and 1998 exclusion rates had been equal? 
Would Individual state scores likely have been lower, if they had excluded the same 
percentage of students In the two assessment cycles? 

6, What are the contributions of changes in students with disabilities versus 
students witn limited English proficiency to the changes in exclusion? 
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Before summarizing the findings to date with respect to these questions, it is important to 
point out that the decision to exclude students from the NAEP assessment because of a 
disability or limited English proficiency is not made by the Federal government. Rather, 
staff members from participating schools make these judgements by applying the NAEP 
criteria for inclusion of special needs students. The information that the criteria are 
applied to may be influenced by differences in state and local policies for identifying and 
educating such children. 

Summary of Findings to Dale 

Of the six questions we wish to addrcsB, analyses have been completed for questions 1, 2. 
3, and 5. and initial analysis has taken place to address question 4. Analysis will be 
forthcoming shortly to answer question 6. The findings to dote are summarized below. 

1* What is the pattern of ebaages fa exclusion rates and Is there an emerging pattern 
across all states or b the condition limited to a few states? 

When averaged over all jurisdictions that participated, grade four state NAEP exclusion 

percentages, tike national exclusion percotages, show only a modest increase. However. 

exclusion percentages have increased during this period in a number of specific jurisdictions. 

In 1998 (the most meenUtato NAEP reeding assessment), 14 of the 36 juriaefiotiona excluded 

a greater percentage of students than in 1992 (the first state NAFP r eading M»jimwnfy There 

were no statistically significant decreases. 

2. What are the contributions of changes in students with disabilities versus LEP 
students to the changes In exclusion? 

The exclusion of students with disabilities from time NABP did increase in a number of 
jurisdictions during the 1990's. SD exclusion percentages were significantly higher in 1998 
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thin in 1 994 for 1 0 of the 36 jurisdictions in both assessments and showed decreases in two 
jurisdictions. Increases in LEP execution p erc e nta ges occurred in fewer jurisdictions and 
tended to be smaller than SD exclusion increases. Between 1994 and 1998 LEP exclusion 
rates increased by a statistically significant margin in 5 of the 36 jurisdictions. Most of these 
increases, though statistically significant were fairly small in magnitude. In addition, one 
jurisdiction showed a decrease in the LEP exclusion cate. 

3« What is the aU th tl c e l relation ship between changes in exclusion rates and gains In 
reading scons? 

In general, there was a moderate positive correlation between changes in average state 
NAEP scores and changes in the percentages of students excluded. Jn other words. 
Jurisdictions with larger increases In total exclusion percentages also tended to have 
larger score increases. Comparing 1998 to 1994, moat of the positive association comes 
from tho data from four jurisdictions. 

Separate correlations between state score gains and state changes in exclusion rates for 
this time poriod were determined for students with disabilities and for LEP students. 
These correlations revealed that the overall association between exclusion and scaJo scare 
increases arise primarily from the exclusion of students with disabilities. There is tittle 
relationship between changes in LEP exclusion rates and score gains. The correlation 
between state scare gains and changes in LEP exclusion rates was -.07. 
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4. How weald gates te State NAE? readteg scores barrc boat affected tfeoccisstee rotes 
hadb— eqnafT 

’The first in a aeries of model bated analytes U used to investigate this question. This 
scenario of the model is very conservative and is Intended to illustrate fe magnitude of 
changes under an extreme net of as s ump t i ons: 

• Tbo cxchnkxn rates are feed over 

* The excluded student's scores met the bottom of 41* distribution of seem ftr 
SZVLSPstodcnts. 

The rationale Ifar starting with (Mi model tests on a need to hmstigato the mostextrerao 
«oM&ile effect on NAHP molts mom teg the excluded student* on the HABP Reading 
amesamc&t mold have boon the poorest per fo rm ers bad fey been tested, la feet, if o© 
effects were** be fend under fbb model than Choc would be no need to exrnnteo fbrther 
scenarios. 



The model described above shows that of fe t«ity-«fx states in fids analysis. fe 1994 to 
1 991 trend findings reported in fe i998 HABPXm&ng Import Card hold fer 33 states, 
far fe Report Card, tea states shewed significant Increases 4a image scale scares tan 
199401998. No stats showed sstadslieaUy significant docliao flora 1994 to 1998. Of 
tfam ten reported Increases, eight woe still significant when fe adjusted means worn 
mod. In a rtditini v owe state for which no aigaifiesnt change w«a pr e viousl y reported 
would have shown e significant increase from adjusted 1994 to reported 1998. It is of 
particular interest to consider large sod s tatistica lly significant score gains and large 
Increases in exclusion rates from 1994 to 1998. When adjusted moans were used, the 
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gains appeared smaller for all four states. However, increases remain statistically 
significant for all but one. 

S. Were there differences in 1998 exclusion rates for students with disabilities and LEP 
students who were offered accommodations? 

Of the forty-four participating jurisdictions at grade 4, six exhibited significantly lower 
total exclusion rates in the NAEP samples where accommodations were offered (know as 
the S3 sample) dun in the samples in which they were not offered (called the S2 samplo). 
In several of these jurisdictions, the presence of accommodations decreased exclusion 
rates fay 4 to 6 percent When accommodations were offered exclusion rates appeared 
lower in oil but five jurisdictions of (he remaining jurisdictions. These ap paren t 
differences were not, however, statistically significant Grade 4 exclusion percentages 
for students with disabilities were significantly reduced In 13 of the 44 Jurisdictions. 
Grade 4 LEP exclusion percentages were significantly lower in S3 compared with ihe S2 
samplo in only one jurisdiction. It should be noted that because the NAEP program 
viewed this assessment, as an assessment of Reading in English, the program did not 
produce translated versions ofti\o instrument 

For grade 8, total exclusion rates wxro significantly lower when aooommodations were 

pennittedm 11 of drc 41 jurisdictions. For students with disabilities, again, 11 of the 41 
jurisdictions showed significant reductions in exclusion percentages when accommodations 
were offered. As was the case at grade 4, only one state showed a significant difference for 
LEP students. 
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HmetSkpt 

fa the coming months, NCES wUlconplste (ha second phase efths analysis In which 
additional model-based analytes will bo conducted to cjumrnc the posable impact on the 
ovmHfato score of changes mexctaaioa rates ©Ywtii^ to addition, the seoond phase of the 
analysis will examine toe idatiottfhip between toe seventy of disabilities of oedoded students 
to 109S compared to previous mau n cB ti AlwyNCBS wfll comp l ete tNABP research 
report on mchwonteiM to 1996 matoomatics and aeknceaaniaiieuU and the 1998 leading 
anomto, which will provide valuable tnsigto tato our fitfure procedures and activities. 
Finally, the restitts of ell oftoese analyses will be examined by toe National Institute of 
Statistical Sciences, an independent body, to evaluate the technical metits of our investigation, 
as well as how the Center Should address the issue of etanges to exdustoo rates in its future 
NAEP r epe atin g. 
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II. Raleaae of the 1998 NAEP Reading Remits 



I wiH now review the plana and timetable related to the release of the 1998 NAEP 
Reading results, The results were reported in two parts: first, the National NAEP results 
on February 10, and subsequently, the State NAEP results (for 43 states/jurisdicUons at 
Grade 4 and 41 states^ mi sdiotions at Grade 8) on March 4 th . 

Issue Background 

Essential to the NCES mission as a federal statistical agency is the requirement that the 
Commissioner must release all education data for the agency. This can take the form of a 
publication, a data file, and/or an electronic release. During my tenure of nearly throe 
years as Commissioner, I can attest to the integrity of NCES’ operation In terms of our 
independence in the design, development, analysis and report content of our statistical 
and assessment work. NCES has achieved an outstanding reputation among its 
users/clients for the quality and comprehensive of its data. 

The issue at hand in the release of the 1998 NAEP Reading Assessment results relates 
solely to the presentation of the data at the February 10 th press conference. The 
Commissioner electronically released the National NAEP data on the NCES website at 
10:00 a.m. that morning. However, the fact that someone other than the Commissioner 
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was the first one to announce these findings publicly si (he press conference created a 
perception (hot called into question the Independence and integrity of the data. 

The following is the specific sequence of actions related to both NAEP releases 
which occurred on Febiuary 10 and March 4, 1999. 

National NAEP Reading Results: February 10, 1999 Release 
In November 1998, NCES staff secured the approval of tho National* Assessment 
Governing Board's Co mmitt e e on Reporting and Dissemination for the draft release plan 
fbr the 1 998 NAEP Reading Assessment Beginning in early January, tho Commissi oner 
met with U.S. Department of Education staff from the Secretary's office and tho Office of 
PubHo Affairs to brief them on the upcoming NAEP release and to secure the Secretary's 
review of the proposed release plan. After a series of Interchanges, Roy Trnby, 

Executive Director of NAGB, and I were informed that there was possible White House 
interest in participating in the 1998 NAEP reading release. 

Reports released by NOES are subject to an extensive statistical review process, titled 
adjudication, which is conducted by the Chief Statistician at NCES and involves input 
from both internal (NCES, U.S. Department of Education end NAGB), as well as 
external (independent reviews of outside experts) sources. 

In late January, as NCES was completing its adjudication review of Ihc 1 998 NAF.P 
reading report card, an error was detected by NCES that related to the 1998 State NAEP 
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reading assessment. Fortunately, the error which occurred during the processing by 
Westat, a NAEP contractor, only impacted on the state results , and not on the national 
data. NCES staff then conferred with both ETS and Westat to develop a revised 
production schedule for rc-doing the initial state report. The Commissioner then 
informed both the Department and the NAGB of the proposal to release the 1998 NAEP 
Reading results in two phases: 

• On February 10 th , the National NAEP results for Grades 4, 8, and 12 for 1998, as 
well as trends from 1992, 1994, and 1998; and 

• In early March (later set for March 4 \ the State NAEP results for grade 4 (43 
states) and grado g (41 states), as well as the trend data for grade 4 (1992-1994- 
1998) for those stato that had participated over time, 

NAGB held a special conference call of the Reporting and Dissemination Committee, 
which endorsed the Commissioner's proposal to keep the February I0 ,h release on the 
National NAEP data, and later to report the State NAEP results in early March. In lato 
January the Commissioner informed the stales in a memorandum of the need to delay the 
release of the 1 998 stato NAEP reading results. 

The week of February \*\ the Commissioner received the request from the Secretary's 
Office to add a practicing teacher to the press conference agenda. I checked with the 
Executive Director of NAGB and secured the Governing Board's support to add a fourth 
participant to the February 10 lh press agenda. 
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Late in the week prior to the release of the National NAEP reading results, Maureen 
Treacy, the NCES Communication Officer, and I were alertod by the Department that the 
Vice President’s Office wished to become involved in the February 10 th release. On 
Monday, February 8*\ I participated in the conference call with the staff for the Vice 
President's office, the Secretary's office, and the Office of Public Affairs of the 
Department of Education. I advocated that the event should proceed at 10. *00 a m t (as 
originally proposed by NCBS and approved by NAGS), with the Commissioner opening 
and releasing the results, and followed by die Chairman of NAGB who would present the 
NAGB position on the National NAEP reading results. Then, the Vice President, the 
Secretary $ad others (possibly the teacher) could comment on the results. When 1 left Iho 
meeting, it was my understanding fliat this la what would tolas place on die 10th. I then 
called Roy Truby to inform him of the conference call discussion and to let him know 
that the original approved plan was still in offect. 

Later, 1 learned from Maureen Treacy that the Department had Informed her that the 
agenda had been changed. There was to be a shift in the order of die presenters and a 
delay until 11:00 am for the press conference at which time the Secretary and Vice 
President would open the press conference. Sine© wc had already alerted our clients to 
the availability of the National NAEP reading data by 1 0;00 0 , 0 ., on Wednesday, 
February 10 th , I instructed NCES staff to release the results electronically on tho Web by 
10:00 a.m. that morning to ensure that die Commissioner would fulfill his responsibility 
to releaso tho data. Tho Commissioner** Statement was released electronically and 
presented later that morning by the Commissioner at the February lQ lb press conference. 
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Three poster bonds were also usod to summarized the 1998 NAEP reading assessment 
results. 

State NAEP Reading Results: March 4. 1999 Release 

Following the release of the National NAEP results concerns were raised over tho 
deviation from the release protocol that called for the Commissioner to release the NAEP 
results. The Commissioner received a letter from the NAGB Chairman that expressed the 
Board’s concerns about the recent press conference for release of the 1998 NAEP 
Reading Report Card. A plan for the March 4th release, which reflected this protocol, 
was presented by NCES to NAGB’s Reporting and Dissemination Committee. There 
was unanimous agreement among NAGB members that greater caution and care needed 
to bo exercised in the retcasc of future NAEP results by tho Commissioner and NAGB. 
The March 4 m press agenda reflects this position. 

On March 4th at 10:00 a.m. the Commissioner released the 1998 Slate NAEP reading 
assessment results that sliictly followed the approved plan. The Commissioner’s 
statement was released simultaneously at 1 0:00 tun. at the press conference and on the 
NCES website. As part of tho Commissioner’s presentation of the State NAEP reading 
results, two additional poster boards were used. 

At the March 10*12, 1999 Boardmeeting, there was considerable discussion related to 
the release of the 1998 NAEP reading results. The Commissioner supported the Board’s 
decision to review the release policies and protocols to ensure that future NAEP data 



16 




79 



79 



releases would avoid any inappropriate appearances orprcceptions. At Ihe May 13-1 S 
meeting, the Governing Board took action on this mailer, 
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APPENDIX C - WRITTEN STATEMENT OF MARK D. MUSICK, CHAIRMAN, 
NATIONAL ASSESSMENT GOVERNING BOARD, WASHINGTON, DC 
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Good morning, Mr. Chairman, I am pleased to testify before the Subcommittee today on 
behalf of the National Assessment Governing Board. 

I am Mark D. Musick, the current Chairman of the Governing Board, which sets policy 
for the National Assessment of Educational Progress. I also am president of the Southern 
Regional Education Board, which is based in Atlanta, where I live. 

The National Assessment Governing Board was created by Congress in 1988. It is an 
independent, bipartisan, 26-member group, composed of state and local officials, educators, test 
experts, and business and public representatives. The National Assessment, for which it sets 
policy, is the only program that provides sound, comparable data on what American students 
know and can do in a range of academic subjects. 

The A ssessm en t has been doing this for the nation as a whole, through nationally 
representative surveys, since 1969. NAEP's state-by-state assessments-which also are done as 
representative-sample surveys-started in 1990. Since then, 49 states have participated, on a 
voluntary basis, in assessments of mathematics, reading, science, and writing. In many respects, 
the National Assessment has become the nation’s most valuable and reliable report card on 
student achievement in elementary and secondary schools. 

It is because of NAEP's role, as an independent, external measure of what students have 
learned that the issues before this subcommittee today are so important. The administration and 
reporting of NAEP must be scrupulously honest, consistent, and fair, and the National 
Assessment must be carried out free from political pressures to make American education look 
either good or bad. 

The Governing Board was created to help ensure these important objectives. In fact, the 
blue-ribbon study commission, which recommended .its creation, envisioned the board as a 
means to buffer" the National Assessment from what it called "manipulation by any individ ual , 
level of government, or special interest within the field of education.” Congress incl uded in the 
1988 legislation a structure for NAEP that the commission felt would “supply needed 
and balances and ‘separations of power* for this important and sensitive enterprise." 

The Governing Board's commitment to these objectives remains strong. And they relate 
closely to the issues I was asked to focus on today: first, the release of the 1998 NAEP reading 
results, and, second, the question of whether the reading scores for some states may have 
improved because more students were excluded from taking the test because of disabilities or 
limited proficiency in English. 
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RELEA SE OF THE 1998 NAEP READING RESULTS: POUCY AND PROCESSES 

As you may know, I have served on the Governing Board since it was established in 
1988. I have been appointed to it by both Republican and Democratic secretaries of education- 
William Bennett, Lamar Alexander, and Richard Riley. The current vice-chairman of the Board, 
Michael Netdes, was appointed by both Secretary Alexander and Secretary Riley. 

Since it was established, the Board has tried to make sure that the release of NAEP results 
is straight-forward and separated from partisan politics. The Board adopted its first policies on 
this issue in November 1990. And the policies have been followed almost completely in more 
than two dozen data releases since then- with only two important exceptions. The first was the 
premature release of NAEP reading results by President Bush in May 1992 at a school visit and 
then a political fund-raiser in Atlanta. The second was this past February when Vice President 
Gore came to the Education Department to announce NAEP's reading results for the nation. 

In both cases, the chairman of the Board, first my predecessor Richard Boyd, and then 
myself, clearly and publicly expressed our concern that these actions were contrary to the 
Board's policy and could harm the credibility of National Assessment results. After the most 
recent case, the Board reviewed its policy and adopted revisions at the Board meeting of May 15. 
The new policy makes it even more explicit that NAEP results are to be released by the 
Commissioner of Education Statistics in an independent and non-political way. 

Since February, there has been one more major NAEP data release-the 1998 state NAEP 
results, which were reported on March 4. That release completely followed the Board's policy 
and the Board-approved plan for the data release. 

I would like to submit for the record the letters written by Richard Boyd and myself 
along with a copy of the Policy Statement on Reporting and Dissemination of National 
Assessment Results, as amended and adopted by the Board unanimously on May 15. 

You will notice that both of the letters are addressed to the Commissioner of Education 
Statistics. That is because of the rather complicated structure of the National Assessment 
program. The Governing Board sets policy for NAEP, and under the law this includes the test 
content frameworks, the student performance standards, the methodology of the assessment, and 
the "guidelines for reporting and disseminating results." 

However, the NAEP program is carried out or administered by the Commissioner of 
Education Statistics who is head of the National Center for Education Statistics, which is part of 
the Department of Education. By law, in setting its policies, the Governing Board is 
. "independent of the Secretary and the other offices and officers of the Department [of 
Education]." The Commissioner implements Board policies, but he is an officer of the 
Education Department, appointed by the President and subject to confirmation by the Senate. 
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Going back to 1990, the Board's policy has stated that NAEP results are to be "issued by 
the Commissi oner... in accordance with NCES clearance and review procedures" and that the 
reporting and dissemination of NAEP results "shall be apolitical... (and) insulated from partisan 
political considerations and processes." Another important point of the policy has been that 
any initial public release of NAEP data shall.be made only in accord with a specific plan and 
timetable approved in advance by the Board or one of its duly constituted committees." 

Unfortunately, the plan approved by the Board was not followed in the case of Vice 
President Gore's presentation at the February 10 reading release. As I said in my letter, "the 
format, tone, and substance of that event was not consistent with the principle of an 
independent, non-partisan release of National Assessment data...." I don't think there was 
anything sinister in what happened, but clearly the release got away from NCES and the 
approved plan. 

In revising the NAEP release policy earlier this month, the Board added new language 
which underlines our previous position. It reads: "The initial release of NAEP data shall be 
independent and apolitical, separated clearly from other programs and policies of the 
Department of Education." It also adds that the official press statement or release announcing 
NAEP results shall be issued by the National Center for Education Statistics, rather than by any 
other part of the Department of Education. 

Commissioner Forgione has told the Board that he supports these changes. We welcome 
that, though, of course, we are disappointed that he will be leaving his post soon. We hope that 
whoever is appointed as the new Commissioner will support the Board's policy and follow it 
too. 



Our view is that. NAEP information absolutely has to be viewed as not political. 
Otherwise, it loses its value. NAEP releases should place the facts before the American public 
and not become political events. 



EXCLUSION OF STUDENTS FROM NAEP SAMPLES 



The second issue we are discussing today is the exclusion of students from the state 
NAEP samples. Commissioner Forgione has made an initial report on this, based on a study by 
Educational Testing Service. I know he has shared the report with the subcommittee and will be 
discussing it with you today. 

However, the issues involved and dilemmas we face go well beyond .the. National 
Assessment program. They are part of 'something much larger going on in American- society 
and affect all testing programs and the efforts in many states and school districts to have high 
student performance standards and accountability. 
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Briefly, the movement for high standards, which is coupled with standardized tests to 
show whether students have met them, is occurring at the same time as efforts to expand services 
for disabled students and to test them under non-standard conditions with accommodations. 
Throughout the country, in many state testing programs and college admissions tests, the most 
common of these accommodations is additional time. Other modifications include testing in 
small groups, reading questions or directions aloud, giving word lists or dictionaries to the 
students, or providing them with a scribe to record their answers. 

One important factor in these developments is IDEA-the Individuals with Disabilities 
Education Act, passed by the Congress with overwhelming majorities in May 1997. The law 
provides specifically that students with disabilities are to be "included in state and district- wide 
assessment programs with appropriate accommodations." Another provision . states that the 
individualized education plan (or IEP), which is required for each disabled student, must include 
a statement of any "modifications in the administration of state or district-wide assessments of 
student achievement that are needed in order for the chil d to participate" in the assessment. 

The NAEP inclusion policy, which was changed somewhat for the 1998 assessments, 
says that NAEP should follow each child's special education plan. And if the plan for a 
particular child calls for accommodations that NAEP doesn't offer, then the student. should be 
excused from taking the assessment. Of course, these individualized education plans are affected 
not only by federal law but also by each state's own laws and policies related to its own 
educational accountability program. And all the laws and policies are open to interpretation by 
the counselors and teachers who deal with particular children in the thousands of schools where 
testing takes place. 

According to a number of surveys and studies, there are. significant differences between 
states and even among districts and schools in the same state about which children are classified 
as disabled and which conditions merit certain accommodations. There also are concerns that 
results from tests taken with accommodations may not be comparable to results under standard 
testing procedures. 

Around the country the variations are particularly great for the category of specific 
learning disabilities, such as attention deficit disorder. And the learning disabled group is 
growing rapidly. It now makes up almost half of 5.5 million disabled children in American 
schools. 

Inevitably, NAEP is affected by these larger patterns and trends. In dealing with 
inclusion and exclusion issues, the Governing Board has tried to act carefully and responsibly. I 
think the materials on Board deliberations and decisions will show that. But the changes going 
on in the states, as reflected in state programs and laws, and the impact of the IDEA law, do have 
an impact on the National Assessment. These changes do have a bearing on NAEP reports over 
time and on comparisons among the states. 
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NAEP has conducted extensive research on the impact of different inclusion rules and on 
allowing different accommodations. There was a major research study as part of the 1996 
mathematics and science assessments. Unfortunately, the full report on this study has not yet 
been issued by NCES. 

However, based on a preliminary report, made by NCES and ETS, that the proposed 
new exclusion rules for NAEP would have no significant impact on results, the Board agreed to 
adopt them for the 1998 reading assess me nt. In order to maintain a trend, we did not allow any 
accommodations in the official reporting samples. However, we did approve having a second 
sample for research purposes-with a limited number of low-cost accommodations allowed. For 
the 1998 writing and civics assessments, which are new exams, the Board approved using- the 
new inclusion rules along with limited accommodations. 

The Board also approved some c h a ng es in the rules for excluding students with limit**! 
English proficiency. However, as the Commissioner’s report indicates, the increase in 
exclusions for these students was small and occurred in only a few states. 

The texts of the old and the new NAEP inclusion rules have been submitted to the 
subcommittee already. We have also submitted the p reliminar y summary of the research on the 
1996 NAEP assessment on which the Board relied. 

Over the next few months we will be looking carefully at the 1998 writing assessment, 
which includes state-by-state results, to make sure that variations in the rate of exclusions and 
accommodations are clearly reported when these results are mad*, public in late September. We 
will also be considering policies for the assessments in mathem atics and yimce. f or 

.the year 2000. In these assessments it will be very important to preserve the trends, which in 8th 
grade math gp back to the first state-level NAEP in 1990. 

Quite frankly, no matter what rules NAEP sets, there has always been some variation in 
exclusion rates between the different states and in particular states over time. Of course, this has 
some impact on the reliability of comparisons,though, overall, we remain convinced that the 
National Assessment is valid and reliable within the quite narrow margins of error of the survey 
methodology used. NAEP strives hard for fairness and consistency. With the different changes 
going on in different states ami from IDEA, that jot is becoming more Htflunfr 

Mr. Chairman, thank you for the opportunity to testify. I will be happy to . try to answer 
any questions you may have. 
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APPENDIX D - SUBMITTED FOR THE RECORD, POLICY STA TEMENT ON 
REPORTING AND DISSEMINATION OF NATIONAL ASSESSMENT 
RESULTS, NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS, 
NATIONAL CENTER FOR EDUCATION STATISTICS, ATLANTA, GA, 
NOVEMBER 17, 1990 
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NATIONAL ASSESSMENT GOVERNING BOARD 
POLICY 



I. Subject: Reporting and Dissemination: General Policy 

II. Policy Number: 1.6(A) (vi) . 08 

III. Effective Date: November 17, 1990 

IV. Policy Statement: See attached 
v. Accompanying guidelines: None 

VI . References: None 

VII. Distribution: Internal, OERI , NCES 

VIII . Responsibility: Executive Director/Assistant Director for 

. Reporting and Dissemination 

( 91 ) 
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Pi9genlnatlon 



. Board staff shall consult with education, business, and 
civic organizations to gather ideas for making NAEP results more 
useful and meaningful to the public, teachers and school officials, 
and educational researchers. 



16. NCES and Board staff shall work with organizations of 
pa w C i S teachers, school board members, principals. and other 
school officials and personnel to disseminate NAEP results to their 
members. 



17 • Presentations and briefings on NAEP shall be given by 

NCES and Board staff and by Board members to business, labor and 
civic groups, national education organizations, and policy makers 
and staff in federal and state governments. 

18. The NCES Commissioner and staff, NAEP contractor 

personnel, and Coverning Board members and staff shall be 
encouraged to speak about the NAEP program and results at meetings 
and conferences, on radio and television programs, and to newspaper 
and magazine writers. In these appearances they shall try to 
increase awareness and understanding of NAEP among the public 
educators, and government officials. y 

19 • limitations on interpretation, conclusions, and 

recommendations contained in NAEP reports, which are stated in 
Point 2 above, shall apply fully to dissemination materials on NAEP 
prepared or distributed by or for NCES, or by the NAEP contractor 
or grantee. 



Approved: November 17, 1990 

Atlanta, Georgia 
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POLICY STATEMENT OH 
REPORTING AND DISSEMINATION 
Of NATIONAL ASSESSMENT RESULTS 



INTRODUCTION 

As the Nation's Report Card, the National Assessment of 
Educational Progress (NAEP) is an ongoing. Congress iona 1 ly -mandated 
project established to collect data through surveys on the 
educational performance of young Americans. Its primary goal is 
to report objective information on academic achievement to the 
American public, policy makers, and educators at the national, 
state, and local levels. 

By legislation (P.L. 100-2y7) , the National Assessment is 
"placed in the National Center for Education Statistics." It is 
carried out b v t-hf> romm i ione r o f Education Statistics, through 
contracts, grants, or cooperative agreements, under policy guidance 
of the National Assessment Governing Board (NAGB) . The 
Commissioner shall be appointed by the President and confirmed by 
the Senate for fixed four-year terms, commencing June 21, 1991. The 
Governing Board is a bipartisan . legislatively defined, policy- 
making body, appointed Ijy the Secretary of Education in categories 
prescribed by law from among nominees proposed by the Board itself. 

According to the statute, "in the exercise of its functions, 
powers, and duties, the Board shall. ..be i^gjiendent of the 
Secretary and the other offices and officers oT the Department of 
Education." Among its specific responsibilities are: 

(1) developing "guidelines and standards for analysis plans and 
for reporting and d issemi nat ing . * results ; " (2) developing 
"standards and procedures for interstate, regional, and national 
comparisons;" (3) identifying "appropriate achievement goals for 
each age and grade in each subject area to be tested under the 
National Assessment;" and (4) taking "appropriate actions needed 
to improve the form and use of the National Assessment. " 

To carry out these responsibilities the Board hereby adopts 
guiding principles, policies, and procedures for reporting and 
disseminating National Assessment results. 
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GUIDING PRINCIPLES (Reporting) 

1. All National Assessment of Educational Progress results 
shall be reported accurately and with clarity of presentation. 

2. Results shall be made available to the public as 

expeditiously as possible. 

3. Reports of NAEP results shall be straight-forward 

presentations of data, free from unsubstantiated observations, 
opinions, or points of view. 

4 • The reporting Hiccominaf nf 

rp<;uU g shall be^tnsulatftd frrn r-^rirnl r"n-it1i-ri fc inn- 

an d processes. ■ 

5. All reports shall contain clear information on test 

methodology, campling, scoring, and scaling procedures. 

6. Release of NAEP reports shall be carried out in such a way 
as to encourage the widest public attention to NAEP results. 

7. Specialized reports for different audiences and companion 
reports providing commentary and analysis may be prepared with the 
approval of* the Board and the Commissioner. However, preparation 
of such reports shall not delay the general public release of data. 

8. Reports of NAEP data may include information from other 
sources on demographic characteristics and resource inputs that may 
provide context for understanding results. 



GUIDING PRINCIPLES (Dissemination) 

9. NAEP results shall be disseminated through the media and 
through a vide network of education, business, labor, civic, and 
other interested groups so as to obtain broad public awareness of 
the data and their meaning. 

10. Reports on NAEP data and any companion reports shall be 
made available to decision-makers and practitioners at all levels 
of education and governmental activity having an interest in NAEP 
results . 

11. Teachers, principals, and parents shall be key audiences 
for NAEP reports and dissemination materials. 

12. A wide variety of materials, appropriate to various 
audiences, shall be developed to carry out NAEP dissemination. 
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POLICIE8 AND PROCEDURES 



Report Propagation and Content 

1 . Reports of NAEP results shall be issued by tha__ 
Compiss ioner ~n t — Kctrn-^rt-4-m g^atlstjp s in accordance with NCES 
Siarance and review procedures to ensure technical soundness and 
accuracy. 

2. Interpretation of results and conclusions as to cause and 
effect relationships shall be limited in official NAEP reports and 
must be strongly supported by NAEP survey results. NAEP reports 
shall not contain statements on education policy or practice unless 
strongly supported by NAEP data. Reports shall contain the names 
of their authors. 

3. The framework of student achievement levels adopted by the 
Governing Board shall serve as the primary means of reporting 
results for all newly-developed assessments in 1992 and thereafter. 

4. Reports of state-by-state NAEP data shall include 
information on demographic characteristics and resource inputs that 
may provide context for understanding results. In addition to 
information collected by NAEP. survey questionnaires, the contextual 
information may include such data as per capita income, the poverty 
rate for school-aged children, current expenditures per pupil, 
pupil/teacher ratio, and average teacher salary. 

5. Members and staff of the National Assessment Governing 
Board. shall be. included in the peer review process of NAEP reports 
prior to their release. 

6. Each report of NAEP results shall contain a description 
of the role and responsibilities of the National Assessment 
Governing Board and a list of current Board members. 

Public Release of NAEP Results 

7. Public rp-jpartinq ancj, release grnciadnrps for NAEP Shall be 
apolitical, similar to those of other Conor esM ongL^ 
^ata-aatherincT programs . _^ v insuV&tina NAEP f rom parti san pol itical 
considerations, the release procedures should serve to uphold the 
credibility of NAEP data and to encourage the continued voluntary 
participation of schools, school districts, and states in N'AEP 
surveys . 

8. The release date and manner of release of NAEP results 
shall be determined by the Commissioner with the approval of the 
National Assessment Governing Board. Any initial public release 
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of NAEP data shall be made only i n accord with a specific olan and 
timetable approved in ^ fh a » n » frs du j v ~ 
constituted com^yyes . All press briefings accompanying data 
VelfiH^e shall be part: of the approved plan. 

9. At least ten days prior to public release the Board shall 
receive review copies of NAEP reports and any other material the 
Commissioner plans to include with the public release. However, 
in the case of state-level assessment results, state identities 
may be masked. 

10. At press conferences for the release of NAEP results, the 
Commi ssioner of Education Statistics and/or his designees (NCES 
***** or representatives of the NAEP contractor or grantee) shall 
P,r^,.g_e.n t^jrta 1 or data f jnd \ npfi ■ The National Assessment Governing 
Board shall select members to participate in each press conference 
to provide commentary about the meaning of the test results. in 
addition, either the Commissioner or the Board may invite other 
officials or experts to comment on the significance of results at 
the time of their release in accord with t-h* ^oecif jc plan approved 
In hy d r- onp of vEs dulV ^ons t ft ut comm i t teeff^ 
Any press statement or release shall indicate clearly that data are 
being released by NCES. 

11. With the approval of the Board and the Commissioner 
companion reports providing commentary and analysis of NAEP data 
may be released by NCES or NACB at the same time as the official 
NAEP reports. Such companion reports may be more speculative and 
interpretive than the official NAEP reports; their authors and 
sponsorship shall be clearly stated: they must be clearly 
distinguished from official NAEP reports. No reports or 
publications may included in the press packet or dissemination 
^Aterials accompanying release of a NAEP report without approval 
of the Board and the Commissioner. 

12. Press conferences and press releases for NAEP reports 
shall not be used to release or distribute any reports that do not 
pertain to NAEP. 

13* Copies of NAEP reports may be provided on an embargoed 
basis prior to release to federal and state officials, educational 
organizations and researchers, and members of the press. 

14. The Board or its Executive Director may issue press 
statements commenting on results at the time NAEP reports are 
released . 
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APPENDIX E - SUBMITTED FOR THE RECORD , LETTER FROM MARK D. 
MU SICK, CHAIRMAN, NATIONAL ASSESSMENT GOVERNING BOARD, 
WASHINGTON, DC TO DR. PASCAL D. FORGIONE, JR, COMISSIONER OF 
EDUCATION STATISTICS, WASHINGTON, DC, FEBRUARY 18, 1999 
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«™* ,wn0MS National Assessment Governing Board 




National Assessment of Educational Progress 
February 18, 1999 



Dr. Pascal D. Forgione, Jr. 

Commissioner of Education Statistics 
555 New Jersey Avenue, N.W., Room 4CQ 
Washington, D.C. 20208 

Dear Pat: 

At the direction of the Executive Committee of the National Assessment Governing 
Board, I am writing to express the Board’s concern about the recent press conference for 
release of the 1998 NAEP Reading Report Card. We believe that the format, tone, and 
substance of that event was not consistent with the principle of an independent, non- 
partisan release of National Assessment data, an important and long-sta n ding Board policy. 

You and I discussed this problem immediately after the release on February 10. I 
expressed my concern that the way the event unfolded would be perceived, understand- 
ably, as an ann o uncemen t of results by the Vice President, rather than by the Commis- 
sioner of Education Statistics, and that there were some significant political overtones. 

The purpose of this letter is not to place blame. We do want to underscore the 
need to carry out the Board’s policy, adopted in 1990, that NAEP results are to be "issued 
by the Commissioner... in accordance with NCES clearance and review procedures’ and 
that the reporting and dissemination of NAEP results "shall be insulated from partisan 
political considerations and processes.' Unless that is done, it eventually won’t matter how 
much attention is paid to the results; people won’t believe them. 

At its November meeting, the Board approved an NCES plan for a February press 
conference to release the reading results. Unfortunately, the plan was not followed (please 
see attachments), and the arrangements seemed to put more focus on the Vice President and 
the Administration’s education plans than on the NAEP report. In fact, the presentation 
of data by the Commissioner took place toward the end of the program, after the Vice 
President and much of the audience had left the hall. The press was placed in a roped-off 
area at the rear, with education association representatives and Education Department staff 
at the front. As The New York Times noted, the event was "more political than usual" 
for a NAEP release. 

Certainly, we appreciate that the Vice President’s interest adds to public awareness 
of NAEP results. For over a decade, Secretaries of Education from both political parties 
have participated in NAEP data releases in appropriate ways. We are grateful for the 
support that the Administration has given to many of the Board's positions. 



too North Capitoi Street, N.W. 



Stdu S2S, MaUttop 7 St 3 
Washington, D.C 20002-4233 
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By statute, however, the National Assessment Governing Board is independent and 
bipartisan. It is strongly committed to maintaining the National Assessment as a reliable, . 
independent report on student achievement. This outlook is in line with the 
recommendations made more than a decade ago by the Alexander-James Commission, on 
which you served, and that formed the basis for the law establishing NAGB in 1988. 

The Board takes seriously its responsibility under the statute to develop guidelines, 
standards, and procedures for reporting and disseminating NAEP comparisons and results.. 
Our members believe strongly that release of NAEP data must be separated fully from 
partisan politics. In 1992 Richard Boyd, one of my predecessors as chairman of the Board, 
sent a letter similar to this, expressing concern about actions by President Bush that the 
Board believed were an inappropriate release of NAEP data in a partisan setting. Since 
then, the role and value of the assessment in national education policy has risen even more, 
making it all the more important that public confidence in NAEP be maintained. 

The pertinent parts of the Board policy on public release are as follows: 

"Reports of NAEP results shall be issued by the Commissioner of Education 
Statistics.. ..Public reporting and release procedures for NAEP shall be apolitical, similar to 
those of other Congressionally-mandated data gathering programs. By insulating NAEP 
from partisan political considerations, the release procedures should serve to uphold the 
credibility of NAEP data and to encourage the continued voluntary participation of 
schools, school districts, and states in NAEP surveys.... Any initial public release of NAEP 
data shall be made only in accord with a specific plan and timetable approved in advance 
by the Board or one of its duly constituted committees." 

I know you share the Board’s belief that the release of NAEP data must be made in 
a non-partisan, even-handed way that assures full confidence in its integrity. At our 
meeting in March, the Board and its Reporting and Dissemination Committee hope to 
discuss with you what further steps should be taken to achieve this important end, and to 
prevent a recurrence of the way the reading release was handled. 

Sincerely, 

Mark D. Musick 

Chairman 



Attachments 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
NOVEMBER 6, 1998 

APPROVED UNANIMOUSLY BY NATIONAL 
ASSESSMENT GOVERNING BOARD - 
November 21, 1998 . j 

NCES RELEASE PLAN FOR UPCOMING NATIONAL 
ASSESSMENT OF EDUCATIONAL PROGRESS (NAEP) REPORT 



NAEP 1998 Reading Report Card for the Nation and the States 



SUMMARY 

The National Center for Education Statistics (NCES) proposes to release the NAEP 1998 
Reading Report Card for the Nation and the States at a press conference in Was hin gton, D.C., 
in early February, 1999. In addition to both the national and state reports, a Popular Report ■will 
be released at that time. Pascal D. Forgione, Jr., Commissioner of Edu ca tion Statistics, will 
present the main findings of the report Comments will be offered by the Secretary of Education, 
as well as by a member of the National Assessment Governing Board. 

The 1998 NAEP Reading Assessment provides a broad examination of fourth, eighth, 
and twelfth grade students’ performance in reading, as well as information about home 
and school variables. The assessment also provides results on the reading achievement of 
fourth and eighth graders in participating jurisdictions. Results are presented for the 
nation and participating jurisdictions by demographic subgroups for both public and 
nonpublic schools. In addition, the reports indicate trends from the 1992 and 1994 NAEP 
reading assessments. 



DETAILS 

Report release: Press conference in early February, 1999, in Washington, D.C. 

Week before release: . NCES will provide embargoed briefings for die Secretary of Educati on, 
the Assistant Secretary of Education, and NAGB representatives . 

Two days before release: NCES will provide embargoed briefings for repres entativ es of 

Governors and congressional staff 

Day of the release: The press conference will begin with introductions and a data 

presentation by Commissioner Forgione. rnmmwits will be 
offered by Secretary Riley and a member of the National 
A ss es sment Governing Board. The press will be invited to ask 
questions. Staff of NCES and ETS will be available to answer data 
questions, if needed. 

Day after release: NCES will brief education associations and Department s taff 
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Attachment 2 



NATIONAL CENTER FOR EDUCATION STATISTICS 
U.S. DEPARTMENT OF EDUCATION 



Press Conference on the Release of National Reading Data from the 

NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS 

(NAEP) 

February 10, 1999 11:00 a. m.'(EST) 

U.S. Department of Education 
Auditorium 



Richard W. Riley 
Secretary of Education 

Emelie Parker 

Teacher, Bailey’s Elementary School for the Arts and Sciences 

Fairfax, Virginia 

The Honorable A1 Gore 
. Vice President of the United States 



DATA PRESENTATION 



Pascal D. Forgione, Jr. 
Commissioner of Education Statistics 



COMMENTARY AND OBSERVATIONS 



Mark D. Musick 

President, Southern Regional Education Board 
Chair, National Assessment Governing Board 



NOTE: At the invitation of Secretary Riley, Under Secretary Marshall. Smith 
commented on the data after Mr. Musick and responded to reporters' questions. 

MEDIA QUESTIONS 
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APPENDIX F - SUBMITTED FOR THE RECORD, LETTER, FROM RICHARD A. 
BOYD, CHAIRMAN TO MR. EMERSON J. ELLIOTT, ACTING 
COMMISSIONER, NATIONAL CENTER FOR EDUCATION STATISTICS, 
WASHINGTON, DC, JUNE 3, 1992 
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June 3/ 1992 



Mr. Emerson J. Elliott 
Acting Commissioner 

National Center for Education Statistics 
555 New Jersey Avenue/ N.W./ -Room 400 
Washington/ D. C. 20208 

Dear Emerson: 

I am "sure that you were ras -surprised and concerned as 
.1 was last week when President Bush prematurely described some 
of the findings of the ,1990 NAEP Reading Report on the evening 
prior to the day when the data were to be officially released 
by your office. It is in that regard that I write to you today. 

As you know / results of the National Assessment of Educational 
Progress are released by the Commisaioner of Education Statistics 
under, the policy guidance of the national Aasessment Governing 
Board. The Board policy/ adopted November 17/ 1990/ provides 
that "the reporting and dissemination of National Assessment 
results shll be insulated from^partisan political considerations 
and processes. ■ 

In accord, with that policy/ the Board approved an NCES 
plan for release of the 1990 NAEP Reading Report at a press 
conference in Washington. I was .surprised and disturbed/ therefore/ 
to learn .that on the day before the scheduled presa conference 
the main results in the report were released prematurely by 
.President" Bush. .According to press reports/ the President 
described the NAEP findings and discussed them during two appearances 
in Georgia: at a private school and at a Republican fund-raising 

dinner. A, summary of results. was distributed* by the- White 
House. 

Of course/ we are pleased by the President's interest 
in NAEP and realize that his attention adds greatly to public 
awareness of NAEP results. We are grateful for the support 
that he and Secretary Alexander have given to many of the Board's 
positions. 

By statute/ however/ the Governing Board functions independently 
of the Secretary of Education: its membership is bipartisan. 

Its general outlook on the need for a responsive but strictly 
independent National Aasessment is in keeping with the recommendations 
made in 1987 by the Alexander-James Study Group on NAEP/ chaired 
by Lamar Alexander when he was Governor of Tennessee. 

( 105 ) 
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Mr. Emerson J. Elliott 
Page two 

The Board takes seriously its responsibility under the 
statute to develop "guidelines and standards ... for reporting 
and disseminating (NAEP) results." Our members believe strongly 
that release of NAEP data must be separated fully from partisan 
politics. This is especially important because of the need 
to assure public confidence in the integrity of assessment 
results/ a concern that I know you share. 

The pertinent parts of the Board policy on public release 
are as follows: 

"Reports of NAEP results shall be issued by the Commis- 
sioner of Education Statistics Public reporting and release 

procedures for NAEP shall be apolitical/ similar to those of 
other Congressionally-Mandated data-gather ing programs. By 
insulating NAEP from partisan political considerations/ the 
release procedures should serve to uphold the credibility of 
NAEP data and to encourage the continued voluntary participation 
of schools/ school districts/ and states in NAEP surveys ... .Any 
initial public release of NAEP data shall be made only in accord 
with a specific plan and timetable approved in advance by the 
Board or one of its duly constituted committees." 

On numerous occasions you have noted the fundamental impor- 
tance of the integrity of NAEP data/ as well as that of all 
reports of the National Center for Education Statistics. I 
fully agree. Therefore/ I seek your continuing diligence in 
doing everything within your power to assure. that Board : policies 
aimed at fostering this integrity will be strictly followed. 

I realize that whenever reports are pre-released to a selected 
group of officials/ there follows the opportunity for the breaking 
inadvertently or otherwise— of your embargo. 

Certainly/ there should be widespread comment on NAEP 
results, but only after they have been officially released 
by your office. The assessment is the nation's most valuable 
source of data about academic achievement in its schools. 

This role makes it all the more vital that release of NAEP 
data be made in a non-partisan, even-handed way that assures 
public confidence in its integrity. 




Yours sincerely 



Richard A. Boyd 
Chairman 



Enclosure 

(full text of NAGB policy 
on reporting and dissemination) 



RAB/js 
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APPENDIX G - SUBMITTED FOR THE RECORD, LETTERS FROM 
CHARIMAN PETE HOEKSTRA, SUBCOMMITTEE ON OVERSIGHT AND 
INVESTIGATIONS, COMMITTEE ON EDUCATION AND THE WORKFORCE 
TO DR. PASCAL D. FORGIONE, JR, COMMISSIONER, NATIONAL CENTER 
FOR EDUCATION STATISTICS, WASHINGTON, DC AND MR. MARK D. 
MUSICK, CHAIRMAN, NATIONAL ASSESSMENT GOVERNING BOARD, 
WASHINGTON, DC, JUNE 2, 1999 WITH RESPONSES FROM EACH 
RESPECTIVELY DATED JUNE 14, 1999 AND JUNE 9, 1999 
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SUBCOMMITTEE ON OVERSIGHT AND INVESTIGATIONS 
COMMITTEE ON EDUCATION AND THE WORKFORCE 
U S. HOUSE OF REPRESENTATIVES 

2181 RAYBURN HOUSE OFFICE BUILDING 

WASHINGTON, DC 2051 5-61 00 
June 2, 1999 



Dr. Pascal D. Forgione, Jr. 

Commissioner 

National Center for Education Statistics 
555 New Jersey Avenue, NW Room 4O0F 
Washington, DC 20208 

Dear Dr. Forgione: 

Thank you for your testimony at our May 27, 1999, hearing on the “Review ami Oversight of 
the 1998 Reading Results of the National Assessment of Educational Progress (NAEP) - the 
Nation’s Report Card. - Your views were extremely helpful in understanding the need to 
ensure that NAEP, NAGB and NCES are as independent possible to protect their integrity. I 
will certainly do what I can to strengthen their independence when the Committee reauthorizes 
NAGB and NCES in the coming year. 

We did not have time to address all of the questions we wished to ask of you at the hearing. 
Attached are the questions we would like for you to submit to responses to in writing. If 
possible, we would like these responses by June 8. 

Thank you very much for your efforts to protect the integrity of NAEP and for appearing 
before my Subcommittee. 




Chairman 



Attachment 
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SUBCOMMITTEE ON OVERSIGHT AND INVESTIGATIONS 
COMMITTEE ON EDUCATION AND THE WORKFORCE 
U.S. HOUSE OF REPRESENTATIVES 

2181 RAYBURN HOUSE OFFICE BUILDING 

WASHINGTON, DC 20515-6100 



June 2, 1999 



Mr. Mark D. Musick 
Chairman 

National Assessment Governing Board 
800 North Capitol Street, NW Suite 825 
Washington, DC 20002 

Dear Mr. Musick: 

Thank you for your testimony at our May 27, 1999, hearing on the “Review and Oversight of 
the 1998 Reading Results of the National Assessment of Educational Progress (NAEP) - the 
Nation’s Report Card.” Your views were extremely helpful in understanding the need to 
ensure that NAEP, NAGB and NCES are as independent possible to protect their integrity. I 
will certainly do what I can to strengthen their independence when the Committee reauthorizes 
NAGB and NCES in the coming year. 

We did not have time to address all of the questions we wished to ask of you at the hearing. 
Attached are the questions we would like for you to submit to responses to in writing. If 
possible, we would like these responses by June 8. 

Thank you very much for your efforts to protect the integrity of NAEP and for appearing 
before my Subcommittee. 

ffp' 

l W 

PXTEHOEKSTRA 

Chairman 
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QUESTIONS SUBMITTED FOR WRITTEN RESPONSE 



RELEASE OF SCORES 

1. In your opinion, to ensure the integrity of the dissemination of data, is it necessary for the 
Commissioner of Education Statistics to release the results of National Assessment data? 

2. With regard to the comments the Vice President made to release the 1998 NAEP reading 
results on February 10, 1999: 

• Did the Vice President’s comments reflect - in total - the findings of the National 
Assessment data? 

3. Did you write the statement for the Vice President for the release? Did anyone' from the 
Vice President’s office consult with you about his statement? 

4. Who informed your office that the press conference agenda had been changed and the Vice 
President would be releasing the results before you? Did you object to this deviation from 
NAGB approved release plan? 

5. Who made the decision that the NAEP scores should be released at 10 am even though the 
press conference was at 11 am on Feb. 10? When was this decision made? 

6. Do you know for certain that the scores were actually posted on the web site at 10 am? 

7. Was the press notified that the results would be released on the web site at 10 am? Was 
anyone officially noticed of this? Was the web. address printed on any information about the 
release? 

8. Is there any evidence that any press used the Intemet to obtain the results? 

9. - Was there a press embargo on the release of the results until 1 V am? Who decided that the 

embargo would be until 11 and not 10? 

r 10. Were-you aware that Deputy. Secretary Mike Smith was invited to the release? Who 
- invited him? 



ll. If.he had:not been present, who.would have handled the questions and answers? 

12; Did Dep. Secretary Smith; in your opinion; reflect the complete context of the NAEP 
results (1992-1994)? Or did he focus on the- scores as contextualized by Vice President 
iGore (1994-1998)? 
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13. Should NAEP and NAGB, as well as NCES and, indeed, the education research agency, be 
detached from the Department of Education and made an independent agency? 

14. To your knowledge has anyone in the Department of Education requested or initiated 
additional analyses of NAEP scores in 1998, 1996 or 1994? 

15. When are the scores released to the Department of Education? Is there any policy governing 
when the Department receives the results? 

16. Is there any legislation or policy that would prohibit the Department from delaying the 
release of the results? 



QUESTIONS ON NAEP INCLUSION POLICY 

1. Why did NAEP inclusion policy change in 1996? Were exclusion rates increasing when 
this policy change was made in 1996? 

2. Was this inclusion policy decision initiated by NCES or NAGB? If not, then by whom? 

Who was finally in charge of this decision? What was the role of the Departments’ Equity 
Task Force? Of the Office of Civil Rights (OCR)? Of OSERS (Office of Special Education 
and Rehabilitative Services)? 

3. Did NCES or NAGB ever consider field testing the new inclusion policy specifically for 
the reading test? 

4. Since the new inclusion policy does not appear to increase the proportion of disabled and 
LEP students included in the test, why was it changed for the reading test? Was there any 
risk of compromising the results by changing the criteria? 

5. What was learned from the 1996 inclusion experience? How were these findings put into 
use as plans were made for the 1998 Reading Assessment? 

6. When can NAGB expect to receive from NCES the analysis of the 1996 NAEP tests, which 
tested the new inclusion policy? 



QUESTIONS ON THE EFFECT OF EXCLUSION ON THE 1998 NAEP READING 

SCORES 

1. In the final analysis, how accurate are these scores? Are state scores really comparable? 

2. Will any state’s score be changed as a result of your analysis? 
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3. Do schools interpret these inclusion rules in the exact same way? If not, does this affect 
scores? 

4. LEP students are now excluded if they have had less than three years of bilingual 
instruction. Before the policy was changed it was two years. Would this in effect exclude 
more students than the 1994 test and possibly affect scores for states like Texas and 
California? 

5. What proportion of schools are monitored as part of the administration of NAEP? Is this 
the same proportion as the 1994 NAEP? 

6. Why hasn’t ETS released as part of their analysis of the 1998 NAEP 4* Grade Reading test 
the actual scores of states after adjusting them for exclusion rates? 
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U.S. DEPARTMENT OF EDUCATION 

OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 

NATIONAL CENTER FOR EDUCATION STATISTICS 



June 14, 1999 



The Honorable Pete Hoekstra 
Chairman 

Subcommittee on Oversight and Investigations 
Committee on Education and the Workforce 
U.S. House of Representatives 
2161 Rayburn House Office Building 
Washington, DC 20515-6100 

Dear Representative Hoekstra: 

I am responding to your letter of June 2 regarding the follow up questions related to the 
May 27“ hearing. I have attached my response to the 28 specific questions in three parts 
as requested. 

If you need any further clarification, I can be reached at 202-219-1828. 

Best regards. 



Sincerel' 






Pascal D. FoTgione, Jr., Ph.D. 
Commissioner of Education Statistics 



WASHINGTON. O.C. 20208- 
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Subcommittee on Oversight and Investigations 
Committee on Education and the Workforce 
U;S. House of Representatives 
Washington, D.C. 



NATIONAL CENTER FOR EDUCATION STATISTICS 
RESPONSES TO QUESTIONS 
IN THE JUNE 2, 1999 LETTER 



Submitted to: 

Representative Peter Hoekstra 
Committee on Education and the Workforce 



Submitted by: 



PascalD..Forgione,Jr.,Ph.D. 
Commissioner • 

National Center for Education-Statistics 
Office of Educational Research and Improvement 
U.S. Department of Education 



June 14, 1999 
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Release of Scores 



1. In your opinion, to ensure the integrity of the dissemination of data, is it necessary for 
the Commissioner of Education Statistics to release the results of National 
Assessment data? 

Yes, absolutely. 

2. With regard to the comments the Vice President made to release the 1 998 NAEP 
reading results on February 10, 1999: 

Did the Vice President’s comments reflect - in total - the findings of the National 
Assessment data? 

No, the Vice President’s comments did not reflect - in total - the findings of the 
National Assessment data. His comments concentrated on NAEP results from 
1994 to 1998. 

3. Did you write the statement for the Vice President for the release? 

No, I did not write nor review the Vice President’s statement for the release. 

Did anyone from the Vice President’s office consult with you about his statement? 

No, we did not see the Vice President’s comments prior to the February 10, 1999 
press release. 

4. Who informed your office that the press conference agenda had been changed and the 
Vice President would be releasing the results before you? 

The Department of Education’s Office of Public Affairs informed my office that 
the press conference agenda had been changed and the Vice President would be 
opening the press conference at 11 am on February 10*. However, I did not 
expect that the Vice President would announce the results. I was also concerned 
that this departure from the NAGB protocol would not be a good precedent 

Did you object to this deviation from NAGB approved release plan? 

Yes, and I asked for NAGB to be informed of this as well. 
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5. Who made the decision that the NAEP scores should be released at 1 0 am even 
though the press conference was at 1 1 am on February 1 0? 

Since arrangements had already been made to make the national reading data 
- electronically available at 10 am, following the original press conference 
schedule, 1 made the decision to maintain the release time to avoid any possible 
confusion or mistakes in getting these data out electronically that morning. 

When was the decision made? 

Once NCES was informed that the Vice President would attend. 

6. Do you know for certain that the scores were actually posted on the web site at 1 0 
am? 

NCES web site technicians were instructed to post the scores at 10 am. At 
approximately 10:15, we verified that the information was posted on our web 
site. 

7. Was the press notified that the results would be released on the web site at 10 am? 

The press was not notified by NCES. I do not believe that the Department was 
aware that the data would be electronically available at 10 am on the Web. 

Was anyone officially notified of this? 

No. (However, NCES had earlier notified the NAEP clients, beginning in 
January, that these results would be released on February 10 th at a 10 am press 
conference.) 

Was the web address printed on any information about the release? 

Not on anything released in advance by NCES. The Department’s press release 
(which was embargoed until February 10* at 11 am) contained the following 
statement: 

“Single copies of the report are available by calling 1-877-4EDS-PUBS (1-877-433- 
7872). The report also will be available on the Internet at http://nces.ed.gov/. ” 

8. Is there any evidence that any press used the Internet to obtain the results? 

.. The press was not directed to the.Intemet until after the press conference. 

' However, there is no way ofidentifying who obtains information from our web 
site. 
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9. Was there a press embargo on the release of the results until 1 1 am? 

The embargo on the report was until 2/10/99 11:00 am. 

Who decided that the embargo would be until 1 1 and not 1 0? 

The Department and NCES agreed that the embargo on the written material 
being distributed should be moved to 11 am after the press conference was 
changed to 11 am. 

10. Were you aware that Deputy Secretary Mike Smith was invited to the release? 

My office was not aware that the Deputy Secretary was invited to the release. 
Who invited him? - 

I don’t know. 

11. If he had not been present, who would have handled the questions and answers? 
Generally, the Commissioner and/or Secretary Riley answer all questions. 

12. Did Dep. Secretary Smith, in your opinion, reflect the complete context of the NAEP 
results (1992-1994)? Or did he focus on the scores as contextualized by Vice 
President Gore (1994-1998)? 

Like the Vice President, the Deputy Secretary concentrated on NAEP results 
from 1994 to 1998. 

1 3. Should NAEP and NAGB, as well as NCES and, indeed, the education research 
agency, be detached from the Department of Education and made an independent 
agency? 

I do not feel qualified to speak on the appropriate location of NAGB or the 
Department’s research functions. With respect to NCES, I have been discussing 
the issue with Department staff in preparation for the NCES legislative proposal. 
Since the legislative proposal must first be cleared by OMB, I will respond 
substantively to this question in a separate statement. As requested by Rep. 
Castle at the hearing, I will be providing recommendations for your 
consideration in the following areas: authority and duties of the Commissioner, 
personnel authority and appointment of employees, authority to publish and 
more flexible use of federal appropriations. 
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14. To yourknowledge has anyone in the Department of Education requested or initiated 
additional analyses of NAEP scores in 1998, 1996 or 1994? 

On occasion, the Department does conduct analyses or requests that NCES 
conduct analyses of released NAEP data. For example, following the release of 
the 1994 NAEP Reading results, the Department asked NCES to look into the 
drop in the average national reading scores among 12 th grade students between 
1992 and 1994. In addition, the Department’s Planning and Evaluation Service 
(PES) regularly analyzes NAEP data as part of Its monitoring of the Title I 
program and Its impacts. 

15. When are the scores released to the Department of Education? 

NCES does not release data to the Department of Education. When the 
Commissioner releases the NAEP reports, the data become. available to all 
interested parties, both within and outside government 

Is there any policy governing when the Department receives the results? 

Yes. There are procedures and practices used during the formal 
adjudication/review of all NCES reports. Materials are shared with reviewers 
under a confidential arrangement that provides independent reviews and 
feedback to NCES on reports under development In this fashion, NAEP reports 
are similarly reviewed by Department staff prior to the formal release. 

16. Is there any legislation or policy that would prohibit the Department from delaying 
the release of results? 

No, I am not aware of any legislation that would prohibit the Department from 
delaying the release of results. 
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Questions for NAEP Inclusion Policy 

1 . Why did NAEP inclusion policy change in 1 996? 

NAEP policy has always been to include as many students as possible. Students 
with disabilities (SD) and limited-English-proficient (LEP) students were always 
included in the NAEP samples. 

In the mid-1990’s the Department of Education began to place more emphasis 
on the participation of SD and LEP students in state assessments. The 
reauthorized Individuals with Disabilities Act (IDEA), for example, requires that 
all students be assessed in some form. 

To make NAEP more representative of the nation’s students, NAEP needed to 
take steps to increase the participation of these students. NAEP inclusion policy 
did not change, but the objective of increased inclusion took on greater priority. 
To accomplish this objective, in 1996 the criteria for inclusion provided to local 
schools were changed to make them more realistic and more objective, and 
accommodations were offered to a sub-sample of students for the first time. 

Were exclusion rates increasing when this policy change was made in 1996? 

Exclusion rates were increasing at the time these changes were made. For 
example, in the 1992 reading assessment, 8 percent of students were excluded at 
grade 4, while in the 1994 reading assessment, 9 percent were excluded. 

2. Was this inclusion policy initiated by NCES or NAGB? If not, then by whom? Who 
was finally in charge of this decision? What was the role of the Departments’ Equity 
Task Force? Of the Office of Civil Rights (OCR)? Of OSERS (Office of Special 
Education and Rehabilitative Services? 

The main impetus for the changes in NAEP inclusion policies came not from the 
program or the program staff but rather from inside the Department of 
Education and the federal government The old NAEP inclusion criteria were 
viewed by the Office of Civil Rights, OSERS, OBEMLA (Office of Bilingual 
Education and Minority Languages Affairs), and the Office of General Counsel 
(OGC) as too open to misinterpretation and misuse. In addition, not offering 
accommodations was considered a barrier to the participation of SD and LEP 
students. The Equity Task Force spurred NAEP to become more proactive in 
increasing inclusion of these students. 
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After a series of meetings with other departmental offices, NCES decided to use 
new criteria for inclusion at the local school, as well as to provide 
accommodations. NAGB endorsed the 1996 NAEP design in which these 
changes would be tried on an experimental basis for sub-samples of students. 

3. Did NCES or NAGB ever consider field testing the new inclusion policy specifically 
for the reading test? 

In the 1997 field test for the 1998 reading assessment, various accommodations 
were tested for their feasibility. Because the field test employs a small non- 
representative sample, it was not possible to use the field test to anticipate the 
effect of overall exclusion rates in the main assessment 

The 1995 field test and the 1996 assessment served as background for the 1998 
assessment The new inclusion criteria and the use of accommodations were 
fully tested in those assessments. The experience gained thereby was used to 
formulate the 1998 assessment design. 

4. Since the new inclusion policy does not appear to increase the proportion of disabled 
and LEP students included in the test, why was it changed for the reading test? Was 
there any risk of compromising the results by changing the criteria? 

The new criteria and the provision of accommodations, first used in 1996, were 
incorporated into NAEP to further the departmental objective of greater 
inclusion. The 1996 design permitted the evaluation of the new procedures. It 
was concluded that changing the criteria did not have a significant effect on the 
inclusion rate, while provision of accommodations increased participation of SD 
and LEP students. These changes do not risk compromising NAEP results when 
a new assessment is being implemented. Accordingly for reading, a trend 
assessment, we used a special sample of students who were offered 
accommodation but whose scores were not included in the official report 

5. What was learned from the 1996 inclusion experience? How were these findings put 
into use as plans were made for the 1998 Reading assessment? 

The 1996 inclusion experience indicated that the new inclusion criteria might 
result in some increased exclusion of LEP students, although the data were not 
conclusive. No increase in exclusion of SD students was encountered in 1996. 
Therefore, it was decided to use the new inclusion criteria for the reporting 
sample in the reading assessment, and to offer accommodations but not include 
data for accommodated students in the reported results. Research has further 
indicated that neither the new criteria nor the provision of accommodations had 
an effect on overall average scores. 
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6. When can NAGB expect to receive from NCES the analysis of the 1 996 NAEP tests, 
which tested the new inclusion policy? 

Results from the 1996 NAEP field test have been shared with NAGB and others 
in the Department in meetings and memoranda, as they became available. Also, 
NAGB staff were included as members of the adjudication team for the “Special 
Needs” report in early May 1999. In that meeting, it was determined that the 
report should be delayed until result from the Reading assessment could be 
incorporated. We anticipate a winter 1999-2000 release date. 
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Questions on the Effect of Exclusion on the 1998 NAEP Reading Scores 

1 . In the final analysis, how accurate are these scores? 

NAEP scores are more accurate than scores from any other source, such as those 
of test publishers. Excluded students represent such a small proportion of the 
student population that their performance usually does not have an impact on 
total state scores. Similarly, post hoc evidence gathered to date indicate that the 
impact of increased exclusion rates on state trend scores is negligible. In 
addition, NAEP can report a known margin of error that describes exactly how 
accurate NAEP scores are. 

Are state scores really comparable? 

Yes, state scores for those states that meet minimal sampling standards are 
comparable. Differences in performance among states and jurisdictions most 
likely reflect an interaction between the effectiveness of the educational 
programs within a state or jurisdiction and the challenges posed by economic 
constraints and student demographic characteristics, including differences in 
exclusion rates. 

2. Will any state’s scores be changed as a result of your analysis? 

No state scores will be changed. 

3. Do schools interpret these inclusion rules in the exact same way? If not, does this 
affect scores? 

NAEP gives all schools the same guidelines, and schools are requested to include 
any students whose capability to participate meaningfully is in doubt. Judgment 
about particular students, and about the interpretation of state practices 
concerning the implementation of IDEA policies can vary. We believe that the 
impact of school differences in interpretation of inclusion rules on NAEP scores 
is minimal. 

4. LEP students are now excluded if they have had less than three years of bilingual 
instruction. Before the policy was changed it was two years. Would this in effect 
exclude more students than in the 1994 test and possibly affect scores for states like 
Texas and California? 

Neither Texas nor California experienced a significant increase in the percentage 
of LEP students excluded from the 1998 NAEP Reading assessment when 
compared to the 1994 assessment Thus, this policy change did not affect NAEP 
scores in Texas and California. 
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5 . What proportion of schools are monitored as part of the administration of NAEP? Is 
this the same proportion as the 1994 NAEP? 

Fifty percent of the schools are monitored as part of the NAEP administration 
for jurisdictions that are new to the State assessment Otherwise, 25 percent of 
the schools in a jurisdiction are monitored. 

6. Why hasn’t ETS released as part of their analysis of the 1998 NAEP 4 th Grade 
Reading.test the actual scores of states after adjusting them for. exclusion rates? 

Much of the analysis that ETS has conducted around this issue represents 
hypothetical assumptions to provide a rough estimate of the possible effect 
There is no way of knowing the true impact of increased exclusion rates on 
NAEP scores because the students in question never took the assessment In 
addition, the National Assessment Governing Board policy prohibits NCES from 
making statistical adjustment to NAEP scores. 
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THE HAJtOWS 
REPORT 
CARO 



m. 



National Assessment Governing Board 

National Assessment of Educational Progress 



June 9, 1999 



The Honorable Peter Hoekstra, Chairman 
Subcommittee on Oversight and Investigations 
Committee on Education and the Workforce 
U.S. House of Representatives 
2181 Rayburn House Office Building 
Washington, D.C. 20515 

Dear Congressman Hoekstrat 

Enclosed are responses to the written questions you asked me 
as a follow-up to the Subcommittee hearing on May 27 . The 
answers were prepared with the assistance of Governing Board 
staff. Where there are Board policies that pertain to the 
questions, the staff and I have attempted to reflect those 
policies. Where Board policies are not applicable, I have in 
some cases given my own opinion as Board chair m a n , though the 
Board may adopt policies in these areas in the future. 

I appreciate the many statements of support for the integri- 
ty and independence of the National Assessment of Educational • 
Progress and its Governing Board, which were made at the 
Subcommittee meeting. 

If you need further information, please contact me or Roy 
Truby, the Board's Executive Director, at (202) 357-6938. 

Sincerely, 

Mark D. Musick 

Chairman 

Enclosures 



too North Capitol Strati. N.W. 

Suit* as. MaOstap 7593 
Washing**, D.C. 20002-4233 
Fho «? (202)357-6939 
Fax: (202)557-6945 
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MARK D. MUSICK, CHAIRMAN 
NATIONAL ASSESSMENT GOVERNING BOARD 

RESPONSE TO WRITTEN QUESTIONS 
From Subcommittee on Oversight and Investigations 
Committee on Education and the Workforce 
U.S. House of Representatives 

June 9, 1999 



QUESTIONS ON RELEASE OF SCORES 

Question 1: Is ' the NAGB policy regarding reporting: and dissemination of National 

Assessment data, a binding regulation or merely, a suggestion to be followed by:the National 
Center for Education Statistics? 

The* Governing Board policy on reporting and dissemination of the National Assessment 
was adopted under provisions of P.L. 103-382, the NAEP authorizing legislation. . Among the 
duties that the law. specifically gives io the Board are: (1) “develop(ing) guidelines for reporting 
r - and disseminating (NAEP) results” and (2) “devek>p(ing) standards and procedures for. interstate, 
regional, and national comparisons.” „ The law also says that the Commissioner of Education 
Statistics shall “carry out” the National Assessment “with the advice of the National Assessment 
Governing Board.” 

' The Board policy has been followed almost completely since it was first adopted in 
"November 1990 with only two important ^exceptions— the reading releases of May 1992 and 
February 1999. However, the authority of. the Board to:make sure its policy is carried out is 
unclear. 



Question 2: Is NAGB required to approve a release plan for National Assessment data? 

• If so, is NAGB required to approve any and all deviations from the originally approved 
release plan? 

There is .no legislative requirement for NAGB to approve a release plan for National 
Assessment data. However, under the Board’s reporting and dissemination policy, “Any initial 
public release of NAEP data shall.be made only in accord with a specific plan and timetable 
approved , in advance by the Governing Board or one of its duly constituted committees.” The 
Board has a clear , expectation that its policies and approved plan will be followed and that any 
proposed deviations from an approved plan will be presented to the Board for approval. 
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Question 3: Who informed your office that the press conference agenda had been changed 
and the Vice President would be releasing the results before you? Did you object to this 
deviation from the NAGB-approved release plan? 

I did not know the order of the presentations at the February 1 0 press conference until I 
arrived at the Department of Education auditorium that morning. As I wrote in my letter to 
Commissioner Forgione, dated February 18, 1 expressed my concern to him immediately after the 
release that “the way the event unfolded would be perceived, understandably, as an announcement 
of results by the Vice President, rather than by the Commissioner..., and that there were some 
significant political overtones.” My letter was sent to the Commissioner on February 18 because 
the Executive Committee of the National Assessment Governing Board had a previously 
scheduled teleconference on February 1 7 and this matter was discussed as part of that meeting. 



Question 4: Were you aware that Acting Deputy Secretary Smith was invited to the release? 
Who invited him? 

Secretary Riley invited Acting Deputy Secretary Smith to speak at the press conference 
after Commissioner Forgione and I finished our statements. 



Question 5: If he (Marshall Smith) had not been present, who would have handled the 
questions and answers? 

Commissioner Forgione, Secretary Riley, and I would have answered press questions, as 
provided for in the release plan. Staff was available from the National Center for Education 
Statistics and Educational Testing Service (the NAEP grantee) to answer data questions that we 
could not handle. 

Question 6: Did Dep. Secretary Smith, in your opinion, reflect the complete context of the 
NAEP results (1992-1994)? Or did he focns on the scores as contextualized by Vice President 
Gore (1994-1998)? 

Acting Deputy Secretary Smith emphasized the NAEP reading score gains since 1994, 
which he said was the “base year” for the Administration’s education programs. 



Question 7: To your knowledge has anyone in the Department of Education requested or 
initiated additional analyses of NAEP scores in 1998, 1996, or 1994? 

This is a question that probably would be better directed to the National Center for 
Education Statistics. I do know that Acting Deputy Secretary Smith noted at the February 10 
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reading release that when he learned NAEP reading scores had dropped in 1994, he asked for 
several additional analyses to determine whether the data were correct. These confirmed the 
accuracy of the results, which were released on April 27, 1995, about two weeks later than the 
April 12 release date approved by NAGB. 

Shortly before the 1994 NAEP long-term trend results were released in October 1996, 
Acting Deputy Secretary Smith asked for additional analyses of trends, using a non-linear (or 
quadratic) approach as well as straight-line, linear analysis. 

Additional NAEP tabulations on children in high-poverty schools have been conducted by 
Educational Testing Service at the direction ofNCES for the Department of Education’s Planning 
and Evaluation Service. These have been used in reports on Title I. Some data were given to the 
evaluation service on an embargoed basis before the February 10 release and were used in a report 
to Congress about two weeks later. 



Question 8: When are scores released to the Department of Education? Is there any policy 
governing when the Department receives the results? 

As noted earlier, the NAEP program is carried out or administered by the Commissioner of 
Education Statistics. The Commissioner is head of the National Center for Education Statistics, 
which is part of the Department of Education. Since the role of the Board is to “formulate policy 
guidelines”, NAGB is not privy to contacts between NCES and other parts of the Education 
Department. 

Under NAGB policy, copies of NAEP reports may be provided to federal and state 
officials on an embargoed basis prior to public release. The policy does not state how far in 
advance this information may be provided. The reading release plan, approved by NAGB in 
November 1 998, stated that NCES would provide embargoed briefings a “week before release” for 
the Secretary and Assistant Secretary of Education. In release plans for some earlier NAEP 
reports, the embargoed briefing was provided two days before public release. Pre-release 
briefings also are given to Congressional staff. 



Question 9: Is there any legislation or policy that would prohibit the Department from 
delaying the release of the results? 

No such legislation exists. Under P.L. 103-382, NCES is established “within the Office of 
Educational Research and Improvement” of the Department of Education. The legislation states 
that NAEP is to be carried out by the Commissioner of Education Statistics. As head ofNCES, the 
Commissioner is part of the organizational structure of the Department 
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Under the Governing Board policy on NAEP reporting, “The release date, time, and 
manner of release of NAEP results shall be determined by the Commissioner of Education 
Statistics with the approval of the Governing Board.** Board policy also sets as a goal that NAEP 
results should be released within six to nine months after testing. However, as noted in the answer 
to Question 1, the authority of the Board to enforce its policy is not clear. 



Question 10. Prior to Fehruary 10, had the Department of Education ever overridden 
NAGB policy? 

Board policies have been followed almost completely in more than two dozen NAEP data 
releases since 1990 with only two important exceptions.. The first, in May 1992, was the 
premature release of NAEP reading results by President Bush at a school visit and political 
fundraiser in Atlanta, The second was this past February when Vice President Gore announced 
1998 reading results in the Education Department auditorium. 

In both cases, the Governing Board clearly and publicly expressed its concern that these 
actions were contrary to the Board policy on insulating NAEP “from partisan political 
considerations and processes.** 



Question 11. What are the most important powers or decisions hearing on NAEP that are 
NOT within the Board’s exclusive control today? 

As I said during the May 27 hearing, there is “shared ambiguity** between the Board and 
the Commissioner of Education Statistics over almost all decisions pertaining to the National 
Assessment of Educational Progress. For the past three years we have worked together 
cooperatively with Commissioner Forgione. The Commissioner has administrative responsibility 
for NAEP and awards and supervises the contracts and grants for NAEP operations. 

The ambiguities surrounding the National Assessment, which go back to the establishment 
of NAGB in 1988, are in contrast to the situation for the proposed voluntary national tests for 
individual students, which are to be based on NAEP content and performance standards. Under 
P.L. 105-277 “exclusive authority over the direction and all policies and guidelines for 
developing’’ these proposed tests is vested in the National Assessment Governing Board. Under 
the current law for NAEP, the Board has “final authority*’ in only one area, “the appropriateness’’ 
of test questions. 
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Question 12: Should NAEP and NAGB, as well as NCES, and, indeed, the education 

research agency, be detached from the Department of Education and made an independent 
agency? 

The Governing Board has always supported independence for NAEP under the direction of 
a bipartisan, independent, and widely-representative board. The law should make it clear that the 
Governing Board determines policy direction and does not just give advice. 

At the time of its first reauthorization in 1994, the Board supported the Administration’s 
proposal, which closely followed the original statute establishing NAGB. The law that was 
eventually enacted changed some of those provisions. Frankly, I believe the changes increased the 
“shared ambiguity” in which the National Assessment Governing Board operates. They also 
decreased the Board’s independence and its ability to act as a “buffer,” as envisioned by the blue- 
ribbon panel that recommended its creation, protecting NAEP from “manipulation by any 
individual, level of government, or special interest within the field of ahicatioif” 

The upcoming reauthorization provides an opportunity for the Board to consider its views 
again on the best institutional arrangements for NAEP and NAGB. 

Question 13: During the npcoming NAEP/NAGB re authorization, what are the best ways 
for Congress to build an institutional firewall between NAEP and NAGB on the one hand, 
and political and interest group pressures on the other? 

We believe that NAEP . should , have a strong independent Governing Board with clear 
policy authority. 

The Board’s views on the statute under which it operates were expressed in 1994 during 
the first reauthorization process.- Since then, the Board has sought to achieve its mission under the 
revised statute. When a new reauthorization proposal comes before Congress, l am sure the Board 
-will again express its views on' the laws under which the National Assessment and the Governing 
Board should operate. 
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QUESTIONS ONNAEP INCLUSION POLICY 



Question Is Why did NAEP inclusion policy change in 1996? Were exclusion rates 
increasing when this policy change was made in 1996? 

Exclusion rates for students with disabilities (SD) and limited English proficiency (LEP) were 
not increasing during the 1990s, but they were significantly high. Almost half of SD and LEP 
students were excluded at the time the policy change was made. In part for this reason, and also to 
reflect state practices in testing SD and LEP students, the NAEP criteria for inclusion were 
changed. As students with disabilities increasingly were included in regular education classes, the 
old criterion, based on the percent of time a student was taught in separate classes, became less 
effective and meaningful. 

The new criteria rely on the nature of the curriculum taught to the students and the 
requirements of their individualized education plans (IEP) as the determinant for inclusion, rather 
than the percent of time in regular education classes. For LEP students, the NAEP criterion was 
changed to be similar to that used by states with high concentrations of LEP students. The 
Individuals with Disabilities Education Act, signed into law in 1997, promotes greater inclusion of 
students with disabilities in testing with appropriate accommodations, which NAEP now offers in 
some cases. 



Question 2: Was this inclusion policy decision initiated by NCES or NAGB? If not, then by 
whom? Who was Anally in charge of this decision? What was the role of the Department’s 
Equity Task Force? Of the Office for Civil Rights (OCR)? Of OSERS (Office of Special 
Education and Rehabilitative Services)? 

The Equity Task Force was an impetus within the Department of Education for increasing 
participation of SD and LEP students in NAEP and other testing programs. OCR and OSERS were 
represented on the Equity Task Force. NCES made a set of recommendations for increasing 
participation of students with disabilities and limited English proficiency in NAEP. NAGB 
discussed the recommendations, made modifications, and, in August 1995, approved the final 
plans for the 1996 mathematics and science assessments. These plans included research studies on 
new inclusion criteria and test accommodations to determine their impact on student participation 
and test results. (See materials under questions 3 & 4 in ‘‘Volume II, Responses to Questions on 
1998 Reading Scores and Exclusion Rates,” submitted by NAGB in May 1999.) 

NCES proposed in early 1997 that appropriate test accommodations should be offered to SD 
and LEP students to increase participation in NAEP testing. NAGB agreed in March 1997 that the 
1998 assessments in civics and writing — two new exams— would offer a specified list of 
accommodations to students who usually receive them in their local testing programs. In the 1998 
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reading assessment, where maintaining the trend was important, accommodations were offered in 
research samples, but only students tested under standard conditions were included in the main 
reports of national and state results. However, the new inclusion rules were used throughout the 
1998 NAEP. 



Question 3: Did NCES or NAGB ever consider field testing the new inclusion policy 
specifically for the reading test? 

The changes in the inclusion policy for NAEP were first tried out as part of the 1995 field test 
for the 1996 mathematics assessment. Research studies on the inclusion criteria and on 
accommodations were conducted as a component of the 1996 mathematics and science 
assessments. There was no* field test of the new inclusion criteria specifically, for the 1998 reading 
assessment because NCES believed that the information available at the time from the 1995-1996 
research studies was sufficient. 



Question 4: Since. the new inclusion policy does not appear to increase the proportion of 
disabled and LEP students included in the test, why was it changed for the reading test? 
Was there any risk of compromising the resnlts by changing the criteria? 

The inclusion policy was changed for the 1998 reading assessment for the same reasons it was 
changed for the 1996 mathematics and science assessments— -to better- reflect school practices in 
states across the nation. States increasingly are mainstreaming students with disabilities in regular 
classes and testing students with limited English proficiency only after they receive three years of 
English language instruction. The new criteria were developed in coordination with state officials 
and with the advice of educators at the state and local levels. As a result of the 1995-1996 field 
;\test and research studies, the Board and NCES believed that there would not be a risk of 
compromising the reading scores as a result of changing the inclusion criteria. 

Question 5: What was learned from the 1996 experience? How were these findings pnt into 
. use as plans were made for the 1998 Reading Assessment? 

The. results from the 1996 inclusion studies, which NCES has provided to the Board so far, are 
included m the materials NAGB sent to the Committee in May under questions 1, 2, 4, and 9. The 
results indicate that changing the inclusion criteria had little impact on the proportion of SD and 
LEP students assessed, but that providing appropriate test accommodations did contribute to a 
significant increase in the participation of these students. The scores from testing with 
accommodations are not included in the reported results for the 1998 reading assessment, but a 
research study with accommodations was conducted. NCES plans to release this study in 
November 1999 at the same time as the full report on its 1996 research study. (See question 6.) 
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Question 6: When can NAGB expect to receive from NCES the analysis of the 1996 NAEP 
tests, which tested the new inclusion policy? 

NAGB expected to receive a report on 1996 inclusion and accommodations research in 
May 1999. NCES says the report has been delayed until November. 
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